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Performance Consequence^ of Sex Bias, 

In The Content of Maj6i> ' ' 

Achievement Batteries , 

Introduction » _ . 

i 

^ nwiber of expectations concerning p^erformatice on educational 
and psychological tests are linked to sex stereotypes. Females, for\ 
example, are expected t;o do better on vferbal tests, while males are 
anticipated to do b^etter on mathematical tests, in this vein Maccoby 



(1973) has observed 'that .during- a'ciole^cence,- females score on the 
^ average about a quarter of a standard deviation, higher than males in 
verbal tasks," and Coffm^ (1961) remarked "If^mpn and women make 
comparable scores on a particular test of verbal aptitude, one^is likfely 
to suspect some bias in the sampJLing of either men,'^-or women, or both." 

Such statements reflect long-standing^ and well festab'lished' expectations 

>, * . 

about differential behavior patterns on tests for. the sexea. 

Many of these expectations were substantially formed in the early 
years of tl>4 development of testing as a movement;/ and they tended to . 
be confirmed in the ^data of that period. For example, in presenting 
the first reports of separate verbal tod mathematical scores for the 
Scholastic Aptitude Te'st of the College Entrance ^ExdAinat ion Board, 
Brigham (1932) found females surpassing males on the SAT-Verbal, by 31 
scale points, while males outdid- females on .SAT-Ma theipatlcal by 26 

points« The symmetry of th^se results, together with their consistence 

. J 

with established views of female and male interests > underscored) the 
development^ of the expectations. , , . . 

Increasingly, however,' these expectations are being reevaluated ^ • 

'.( ''■ ' ... 

in t.erms of the cultural influences which may determine them» Whereas,* ttt 
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,was an implicit assumption that' sex differences in t§st 
mehow reflect fundamental -psychological differences 

- • - ■ • / 

exes., it is increasingly evident that the^cultural expec-S 
shelves have operated in ways which tend to bring about the 
ttems. Thus, tliere is a self-fulfilling component to cultural 
s for .sex differences". The increased awareness of this* 
ines with a renewed concern for 4)rinciples 'of social equality 
atte^tion on the implications of se^^dif f erences *in tes^} 



a society whi'ch is formally dedicated to maximizing the develop- 
f individual potential, the proper detefrmlnation of that potential 
be given a high priority^ Whete ^est scores reflect anS sustain 
ural expectations which are in. fact counter, to the realities of 

' •* 

m potential, fehls constitutes an obvious* bias, one which maybe in 
rect. conflict with the ideals of the society/ Given this larger 
dial context*, there is a vital need ^o understand' the facfot^.which 

ontrlbute to the observed^ sex differences. The present study-^^a 

• ' ' ' ^ ' • , , "l 

basic step, toward; securing this understanding/ 



A strong demonstration of the- need ^to investigate sex differences 
<^in test scores is* provided by data 'derived from administrations ot the 
Scholastic Aptitude' Test of » tl?e College Entrance Examination Board. 
The verbal SAT-V and its mathematical counterpart', SAT-M;,"are widely 
ys^d to select indiv^iduals for post secondary, education, often in highly^ 
comp*etitive'^situations* . *T6 the extent that individual options for post- 
secondary education are influenced hy^the SAT^it is indeed a "major 
test." In the Scholastic Aptitude Test-Verbar.of recent years there 



has been virtually no. differences. between the seXes in performance. 

/ • * 

Whereas '40 years ago there, were th'e reported Vemarl-e superiorities* of 31 
scale score points, by the late 196^ 's females demonstrated dnly trivial 
superiorities of 1 to* 3 scale score points (Angof f , . 1971>. • Since the 
scale is so constituted that ^a single standard deviation is set at 100 
points, the. original BrigHam dat^ itiay J)e taken as roughly consistent '^\'- - 
with ^Maccoby's gener^alization^' quoted above. But, a^ Angpff's da'tsa show,' 
the* generalization no longer applies, at least for^this test. The reasons 
* for this !apl)arent shift in performance are not easily identified. 

Test outcomes are the result of an interaction^between test taker 
and test content. The content of tests, however, frequently reflects 

thfe very stereotypical ejtpectations aWut Ssex and identity that prevail 

* * ' > 

in the society. The *^ervasivip effeots of iuch stereotypes are, in fact, 
the fundamental basis for the expectations concerning sex difference 
in test performance. Thus, as in any stereotype, there is an obvious 
circularity, with test content itself serving to reinforce the accep- 
taqce of sterebt^es which may affect test performance. Tittle (1973) 
and Lock^ed^ (1973) have documented th^se'^'stereo types extensively. 

Bias, of coujTse, is present whether or noj^ sex references in test 
"consent actually^.^fetermine score outcomes'. The simple presence of* 



unequal nximbers of rftf^rences to ^each of the 3j^^s> is evidence of content 

* . • V / ^ • ' 

bias* • Cettainly the qualitative distln/:t:lons among the referents^ 

reflecting such sex stereotypes as ''male laborer" and "female hou^^p|:ker" 

constitute another f orfii of bias. Lockheed) (1973^has elaborated dlatinc-r 

tions among these fo'rms of referential bias, pointing to the association* 

between sex and occupational rol^^a.pd sex and intef^sts. 
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Stu<fies/of • t;he impact of gontept^bias on tHe' .relative performance 
of males 'and f^les have beea^reported b/^ly a few inV(&stieat6rs. 

.Coffman (1?61), Donlon (1973), and 'Strassberg--|losSiber^arid Donlon (1975) 
s.tiidied the College Board ScHolast^c Aptitude riest. . Hicks and Donlon 

•(1$76) studied the aptitude tests of the Graduate Record Examination* 

In each'study, a number of demonstrations of th6- impact $f content on 

* . - ' ' . ' ^ ' . 

the relative performances of females and males a,t .the item level were- . ' ' 

reported, both for verbal and mathematical;^ Many pf the verbal 'demon- ' * ^ 

\ ^ • ' ' ' ' , y \ \ / ' ^ ' ' ^ 

•»trations were essentially in agreement Vith the stereotypic*! electa- ^ - 

tions-. Thus, females did relatively better .on Human Relations Tjatf^rd-al, \ 

males on Sciencje or Economics. Within mathematical jnaterial, however/ 

« • • '« 

some of the findings were unanticipated. -Therfe^is afunctional male 
advantage jr^iut It is extensively modulated by corftent. ^ Succinctly 
stated, females do relatively better on algebra than "geometry . Demonstra-. 
tiorrs such as \hese af the iteinolevel do -ftot tn themselves '^^tablislr^ 
the presence of test. bias in the sense of an inequity iit the distribution 
.of scores or of decisions based on scores. BuC they clearly establish ^ 
that the observed score dif f erenceaj>ejtween the sexes are a function in 
part of test eContent specifications, and that the magnitude of such ']i 



content influences may* be surprisingly gre^t.' Donlon- (1973) ^ for example, 

estimates that the oljsfejjved sex differences of approximately 40 sc^le 

^ ' ' ^ • ' ^ / 

^points on College fioard Scholastic Aptitude Test - Mathematical, could 

be reduced to 20 or increased^ to 60, points by shifting the specifications 

with respect to/ the proportlon^of algebtp and. geometry items. 

The. present sttidy .examines tTie content of selected , major achievement 

tests,c analyzing, it for bias In the frequency of references to the s^exes. 



* • ■12. 



Further, ft.eontrasts the^uccess of the pexes on each item, in each' of 
the tests considered, FlJiariy, it surveys the .relationships between • 
■ these two factors , relating test content and the difference between the 
sexes. 



f 



* * \ _ Methodological Considerations * 

A number -pf metho'dological considerations underlie the work reported 

* A, . 

in this study. * The major topics of this natfu.re are reviewed. in the 

following sections. VBriefly, .they are 1) the selection of instruments,* 

2) the preparation of the data bas^ 3) tlie condxict of the content 

^* ' * . " 

analysis^ 4) the relationship of t-he present content analyses to that 



of Tittle, 5) the approach to assessing content in mate'Jlal related to 



a^sociat^ with the items themselves, and 6); the assessmeiit^of coder 
.reliability. * . ' * < / . 

Methodological aspects of other parts of the study, such^as the^ 
"implicit" analysis, ^or the comparison oi^ wrong-answer distribution^ are « 
covered in the 'se^ctions Whicb Veport the results, of these activities.' 

The Selection, of Instruments ^ ^ ' . 

The following criteria governed the selection of instruments for 
this 3tudj^: - \ . . 

1. A focus on achfevement batteries' with a diverse set of content 
offerings.^ Content could not be restricted .to any givea subject 
such as mathematics. , ' . \ 

2. An effprt to select a total set of instruments which would *span 
the entire grad*' range from. 1-12* . A number of researchers 

4 

have reported age as a moderato3;/of sex differences in Interest-- 



and. attainment. The pijesent stliidy, whi-le not designed to 
specifically determine these relationships, sought to compdire 
. , . the sexes at a number of points on the a^e and ^ra^e scale*. 
3. A preference 'for widely used and recognized batteries. In 



^part, this study derives importance from the widespread- use' , 
of ttie instruments if studies, bindings concerning obscure or' 
seldom used Instruments would be less compelling v 
i 4. A requirement that there be available national norms saiuples, 
for a. year more recent than 1965, selected b^ procedures 
with a high level of statistical' rigor. 

5. The need to be able to secure item- level data on the individuals 
in the norms samples. '_ ' ' * 

6. '^Preliminary evidence that test content in the. items was suffi- ^ 
ciently varied,- in th'e s^nse"ofVsCoijta±ninjg teferences to both 
males and females, that the' effects of reference^ to both sexes c 
could be tested. The survey by Tittle (1973) was used , to 
provide this preliminary evidence'. , * • 

The application of the^ criteria led' to the final choices: 

j/' Grades 9-12 Sequential* Tjcste of Educationa-l-Ptogress, 

' t , Series II, Level 2A, and ^ 

^ . California Achievement t4st. Form' A, Level *5 

^ (Mathematical* Problems Subtest onj-y) 

^ Grades 3 - 8"^ .Iowa Tests of Basic Skills, Form 6 ' 

Grades- 1 - 2 ^ . Metropolitan Achievement Tests (Reading only) ' 

The Sequential T^sts erf Educational Progress (STEP)' ajid the Iowa 

Tests of Basic Skills constitutefd major batteries with mirlt~ifest, multi- 
• • * . ' . 

level sampling. The mathematical S.TEP test, /Mathematics Basic- C6ncepts , 



.however, d9es not contain mucH sex referential content. Accordingly/ 

the Mathematics Problems Test t)f the California Achievement Test, 
^Level 5, Form A was .included as a. supplement to the STEP test.. Only 
' this single subtest, Section 5 of '^e CAT battery-, was studied. 
^ The focus on the year 1965 or later as a criterion for the a^pro- * 
priateness of norms was a compromise between ideal samples and the 
realities pf test publishing. Ideally, the more recent the sample, the 
better, foi; a great deal of social- change with respect to sex role 
stereotypes has taken place since 1970 and is now continuing. In such 
circumstances the analyses of performances based on *dat|» gathered in 
1968 or 1970 cannot be/saf ely generalized to the present year. However, 
the opportuiilLy Lu consider nationally representative^ samp'les on major 
batte-ries outweighed the .cons trairits bn generalization to Jjhe present. 



*^ A major decision on the aArai.laBility of information ,_Has_jnade in 
selecting t;he Iowa Test^ of Basic Skills. There was considerable methodV 
ological interest- in the study of responses to distractors or wrong 
answers in. options. However, the item-leyel data- for thelTBS battery 
is only' available in terms of success or failure on the item. « IJie 
decision tq forego dist^ctor information reflected th^ high standing 
of the ITBS on^the other criteria J was^f^^t that adequate method-, 
ological 4/Bmonstratton 'or comparisons inyolving distractors could be 
niade by fixaminipg the STEP tests and the Metropolitan Reading Test* 

Preparation, of the Data Base _ ' 

\. The relevant test piiblishers--CTB-MeGraw Hill, ' Educational Testing .. 
Se?:mce, Harcourt Brace Jovah^ich, and Houghton-Mlf f lih~were approached. 



with requests for data ^om the appropriate notiisXample/ I« each case, 
complete coqperation was extended to, this studJ^ The request from the 
stud^f was for a male and female sample, apr^topriately selected'; with a 
minimum N of 1,000 in each sex. For th/lowa Tests of Basic Skills, 
this request was met;precisely , bec^e the very large Ns potentially - 
available from their total norms/Sample dictated some kind of sampling. 
For the other publishers, howi^er, it proved le.ss expensi^Lto transmit 
copies of the entire narms/tapes^ at tb'e.. appropriate level. This resulted 
in some variation in sample size from one major instrument to another, 
.and. from one .subtest /o anothet within the multitest major batteries. 

In each case, however,- tjtie required minimum of 1,000 cases was obtained. ' 

'A V - ^ ' ^ ■ -^^ . 

. Further, a 44itj^al ^f ormatio n wa3 in somia c^ses sup p lied,^ such as 



both Primary leyel^ of the Metropolitan' Achievement Test. 

i \ ^ . ' \ I 

. The result was a potential for an empirical' description of the 

A ' ' 

^ • / - . • - 

•relative difficulty »f or males. and females in approximately 5,700 itemr 

' *■ 

grade comJ>inations, or apiproximately 11,400 basic item analyses. It 
was felt that thiS'Vas potentially tgo, large a component bf, project 
resources -to "devote to the statistical analysis, particularly since the ' 

— - * ^ - -..•(n 

assimilation of computer ' information from" other systems into the ETS ) / 

" ' ' . ' , '7 

system, requirifig somewhat special reorganization of tapes and .layouts, • 

* ' ^• 

was an unavoidable data processing expense. It was decided to limit ■ 
t^he, analysis to grades 2, 5, 8, and 10. This decision was based on a 

desire to C9nsider the 'problem of sex bias across as broad a range of* 

* * • 

grades as possible. Further, it permitted a, 'maximum use of the item 

content of the lo^^ Tests • ; m the Iowa series adjacent grades take some 

overlapping item contenjj. Grades 5 and '8, however, are sufficiently * * 



,aepay:ate'4 so that no item overlap 'occurred. This increases the total 
number of items evaluated. . * * , 

yitdjnately, .then>, the proj^ect performed approximately 3,000 analy^ses 
uniquely identifiable by item, grade level, and s ex. Thes e item analys es 
formed* the basis for the correlations which constitute the major method- 
ological approach qf the study .^ • ^ 

' * ' \ . ■ ■ • " 

Item Analysis technique . ' ^ 

. Th$ item^ analysis was a standard' program which provided the following 



data: 



* -1. • Frequency ^of selection for each" response category. 
2. percentage of responses for^each category. ^ 



^3; Average score on the test, excllidihg th^ item Itjself , 

fgr each response categpfy. » , . . » \ r 

,4. Standard deviation o| scores on thfe test, excludi4ig 

> * ' • 

' ' the item Ijrself > for each response categpry, 

5.. Point biserial and biserial correlation coefficients ' 

For the STEP and Metropolitan Tests the^ response 'categy3ries we're 

the optional resppnses, whose number depended on the specific character-^ 

istiQS of the item, knd the Qmlt response or Not Reached response. For 

the Iowa Tests only four response categorieys were used: Right, Wrong, 

Omit or Not Reached, - ^ r \^ 

The selection of a meagre of item difficulty wias basecf on* practical 

tad statistical considerations. It is a frequejit practice in-d^m ^ 

analysis to substitute some sort e'f area* transformation fot*the actual * 

percentage, using 'the baseline values for the normal, curve. One such 



V 




transformation, ^or ex^ple, undetlies *the inde^ of d;L.fficulty whidi Is ^ 
used in the standar^item analysis at Educational Testing Service-./ This- 
ind^^, A,yi6 thVtransformationNo^ the observed percentage to tii4 corrV 
. sprondi^g. baseline value in, a normal c;\irve with mean .^13^ and .^and^rd'^ ; " 

xievia^ion 4. ' * ^ - - 

/ / ' ^ ' ^ ^ , 

^ / The/ principal advantage ofOguch a ghange lies in the., t^eoreticjal ' 

^ problems with-'the statistical operations of averaging or, correlation; ' 

- jif the;undeXing distribution is' a- normal one°, the petfient_passing will 
have a skewed sampling distribution. It was felt,. however, tltat; the 
^sampling distribution for the' Item -difficulties in the,, current study was 
Ukely tQ avoid extreme values , simply becaus^ the fe^ts were carefully , 
made- to afford maximum disoriminatlnn. 'Thrmighout the~^ange 'of percent- 



ages, from ,10 to .90, the relationship betweer^ these valuks and a 
^ifarmal t^urve' baseline yalue i«; not markedly non-linear, a point" noted 
by Curelion (1963) and others. In the current study, tl^refore, this 
transformation was not applied. 

Accordingly, the comparisons of , item difficulty were in terms of 
percentages. The base for the percentage correct -may be chosen from 
several options, all with some logical advantages and disadvantages. 
Th^se are: ^ ^ ' • * ^ 

1. The total group of subjects. . ^ 

2^ The tot^l group minus the number not reaching the 'it.em. 



3. The tQtal group:,minus both the number ^aj^reachlng. the Item 
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' . ' - and the njimber omitting the item. 6'^. 

•Vlhe distinctidn between "pot reaching" and ^omitting" is establisTied 
' ' ' i ' • 

in. terms of whether any later item is respond4i to. Thus, if an -item 

it ^ 

is not answered, >ut an item appearing, later' in the test is, then th4 
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>'item is considered to have teen omitted. -But if ho later item, is 
answered, then the i,t^m is considered to ha^_been not reached. In 
ef'fect, this assumes that the last item answered is t?he last item 
, worked on, any later , items j^eing* not reached. It implicitly assumes, 
that blank or null responses prior to the last item are different from 
\ those which follow. They are generally held to be based upon a mental 

prpeess in which the item is read bu5 upon reflection not answered. 
,The later, items without responses, are considered to have beep unread, 
hence not reached. , ^ * * ^ 

^ None of the possible bas^s for t|he* percentage passing ig {Perfectly 

satisfactoiry./ for the problem of 'dropout in* i1tem analysis is unavoidable. 

^ The calculation of percent ^paas based ^upon the number who re^ch the 

It * ' ' " . 

* • \ \ '^ item is likely to provide an underestimate of thie difficulty of the i^em 

M ror^^he, total group, since those who fall to reach an item are typically 



^less able subjects, with a lower than average pr<3babil iky of success. 

On -the otjier hand, using the base of total subjects, regardless of .progress 

through the test, opens the. way to attributing differential kfdm difficulty 

for the sex^s to some content characteristic when in fact it might be due 

* to differences in rate of work. <- , 

There are strategies for adjusting observed it'em^diff icultles for' 

• * . , _ , ^ ^ ' \ ' 

dropout. Thus, the standard 2TS item analysis i which uses a normal 

curve baseline' approach, adjusts the baseline measure by an increment 

or decrement appropriate for the shift in the average score of the group 

which actually responds to't^he item. Such adjjiistments , however, were 

. r^^ considered too elaborate for the current study* Instread, the problem of 

S-S- • Not Reached responses was approached for all items via -a 'correlational 

analysis . ^ \ * / , . . 
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Figure 1 



FEMAtES . 



Sample Item Analysis for Females /Males Callfiomla Achievement lest 



Item If 1 




Difficulty 


■ 0.8275 




Point Blserlal = 0.2504 






\ / 

Not 
Reached 


\ 

Response 
Category 


Total 


1 


2 

0 


. ^ 3 


4* 


5 


»* - . 
Omit 


. 'Out of 
Range 




235 7. € ' 


95.0 


21.0' 




19S7.0^ 


.162. C( 


2..0 


0.0 


\ 

0.0 


Per^'Cent 


« 


4.03 


0 . 89 " 
• • 


5.a9 - 


83.03 


6.87 


0.08 


0.0 


* 0.0 


Mean 


- : 8;101 


5.179 


^ 3.905 - 


7.133 


3.417 


7.253 


8.000 


0.0 


, -0.0 


Sigma • 


3.265 


2.741 


2.223 


2.924 


3.205 


3.100 




0.0 


\ r ' 

■ V 0.0 


* 


MALES 
? 


















Item // i* 

> 


« 


Difficulty 


« 0.8275 ^ 




. Po^t fiserlal - 0.2504 









Responge 
Ca 



Me^n , 
Sigma 



r:^. Total 
2753.0 

,9.194 
3.306 



1 

107.0 
3.89 
6.196 
3.257 



21. 0 
0.76. 
5v714 

• 2.881, 



3 

125.0 
' 4.54 

3.015 



4* 
2278.0 
82.' 75 , 
9.5^2 



5 

221.0 



3.J.99 s 



V 

8.03 

7.928 

3.117 



Omlt^ 

1.0 

-0.04 

4.000 

0.0 



Ou^^ 
Range 

0.6 * ^ 

0.0 ' 

0.0;' 

? ^ 0.0 



Not 
Reached 

o.q C 

0.0 

0.0 , ^ 




Methods of Content^Ahalysls ^ 

^ The? approach to a content analysis sought to develop procedures 
which would be- Useful in° analyzing any multiple-choice instrument. The " 
task of . thfe. content analysis is itb. idei^lfy and to, quantify differences 

' ^ ^r4^ ♦ ' ' ^ ' ' * 

in tests^fi-tK respect to wh^t they say about males and females, and the 
generVL concept of such, a content analy§;Ls ds almost ^nlzuitively clear: 
earlief studies by Tittle (1973) and Lockh'ee^ (1973;)^«1fiave established 
the 'gerferal lines of the method. No earlier par6ject, heaver, had' provided 
a record of the procedures used ft)%. Xhe content analysis which is suffi-' 
ci^ently detailed to constitute a basis for* replication of the study or 
its extension to new materialsL^^nThis project sought to develop a ^record 
of its procedures sufficiently thorough for others to repeat them in some 



detail. 



^ The complexity- of -language-.'is acch;§rienge to any/ &ys|:em which seeks 
to describe it with quantitative analyses. Pronotnr antecedents , ^4or 
example, are often clearer ^o' the* writer^ than to -the' reader; Yet the 
i"4entif ication of t|)e proper antecedent .is obviously necessary when" the 
pronoun has to be cotarf^ed^ as^^part "^of the person-referential conteht of 
the material. Proper names arid mixed-sex plutal groups^' al§o constitute 

f * - . - * 

special problems to a sex-orie"ht«ed content analysis, \ 

- ^ • : • / . 

Added to the problems of language- are problems' of test i^hd* item 
structure;. This study paid specific attention to Iteii. structure, both* 
^s a potential variable fol* additional analyses and. as a characteristic 
which could assist in the rapid i^enti'f -^^^JH^ of differences in coding. 
- Silt item- structure varies from format £0 fon^t. "Stem" and^"options" . 
may def ine a basic item concept>< but there are a numbet of more complex 



situations than this, x?ith response t;hofceS. incdrp^iE^d -directly into" ' 
material and designated by underlining, 'ir with th§ stem and options ° 
only constituting a question in conjunctTSTwith some associated stimulus. 

The content analysis procedures were as- follows: For each item, an 
identification number was assigned ^e^ign^ting the test, section, content 
of the section,-^ and Item number. Each item«was also categjaiizexl as: 

• • (1) ixrimarily, verbal/text, (2) primarily pictorial,, (3) primarily numerical/ 

* . ' ' ' \ 

• symbolic, or (4) mixed. - . • . \ 

s ■ - .. ' ■' ^ ' ^ , 

A test item' is defined 'as having thre'e components: (1) the stimulus' 
► f ' ■ . 

(2) the stem, and (3) the responses. Each of th^se components was 

- , V' - t ' 

analyzed separately, . " , - • 

^ ^hg-£g ntenr an a lyg ls focuse d, on fo u r indic a tor s;, — (1) aVe> n umbers 

of references 'ta malBs and to femalesr, (2) the numliers of ma'Je persons 

.and female persons, . (3.) the various xoles ".attributed" to males and . ~~ " 

females in the content and (4) the relative "status; of mle and* female _ 
roles. ^ . , ' ' 

.* ' « " . ■ '^-j: ■ ^ " ^ 

In order determine the^ relative .balance* o^f males and females, ^ 
the sex of actors must be identified. Four identification procedures 
were specified: 

1. The noun is Inherently' sex-linked, e.g., motheV - 
^ .father, sister , -brother. . > * 

2. The noun is folind to have a sex specific ^definition 

\ . . f- . ' ~ 

in tjie 'dictionary , e.g., 

\ ballerinas X. a principal female dancer in- a. 
ballet, cotapany, .2./any female ballet dancer. 

V- ^ . ' ' . « 

3. The noun is a definite female or male txame, e.g.,' 

^\ > ' 

' ^ : Bill, Mary. - t 
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4, The n^un has a male, female pronoun which 
refers to It, e.g.,- Pat went' to her class? 
The actor (s) -in the item, either single individuals or a group of 
indiyiduals, were then counted. In cas^'es whe« sex could not be 
assigned, a neutral category was used. 

In order jto determine the relative balance of male' to female ^ 
references,- the number of actors plus other worjds such as pronouns which 
refer tp them in the item were' counted;. repetilr^Iis were also counted. ^ 

Words whicl^showed vocations, avocatibns; or special functions of 
peopl.e, (for -example, doctor*, mothejl were :<:oded as roles. Roles*^^ere 
not .inferred from the descriptions of individual behavior. For example, 
the Tole. of "househusband" is not inferred from the sentence, "He 
cleaned ij^e house ahd fixed dinner," The identification of particular 
roles as female,, male,' or neUtral was decided^ bj^^the percentages of 
females and males found actively engaged in that rqXe aS^ documented 
by the Occupational Characteristics ^ 1970 Census of Population >and other 
sources. When 80% or more of the Individuals engaged in aa-- occupation 
were one sex, the occupation was defined a^ a sex-typed role. Other _ 
occupations were classified as neut^tal^oles * Historical consideration 
of roles wtfs handled by general knowledge^. If an item involved a role 
that was generally known as restricted to one sex (e.g.,, -knights, the* 
congressmen of 1800) it was coded as a sex-typed role, ' ' 

Ill order *to^. ass ess relative status; both males , and females ne'ed to 
be^resenft in an, item. Assessing the status of roles in an*item focused 
bn the sex of the actor holding the most superior role"." Thus, the 
status of males and females in an item could be equal, .female superior', 
or male superior, - * / * 
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Comparison of the Content Analysis with that of Tittle ' 
*k * * 

* * . "* • ' - 

The content analysis carried out in the present 3tudy covered tests 

^ach of which had been considered by Tittle et al. in their earlier work. 

The techn^rtlue of the, present analysis was somewhat different, so that 

- ' the resxAts cannot be" strictly compared. Further, the current analysis 

developed^ a statistical ^suiranary irhlch focused on the item leVel, in 

contrast to the statistical description of total tests rep orted by Tittle 

It is^ possible,., however^ to formulate . the. data from the -two studies- s^ 

as to reflect some of -the consistency between them. Such a contrast can 

Nhelp to evaluate the extent toUich the Specific content analysis carried- 

out in this. project might distort cbntent-diff iculty relationships. The 

Tittle r es- u l t st rp f l p rf p d c o nte n^l ^y,' but hav e b een widel y xil^ semiuaLeJ — 



and were th§ first major and systematic* application of content analysis 
to Cjie problems *of;sex bias in tests. If the current analysis was greatly 
different -from the Tittle results^ the generalizability of thi content-- \ * 
difficulty associations couid be limited. 

ThV major element of- the Titile approach was the ratio- of male refer- 
ences to female^ references, considered in two classes: "all" and- "regular, " 
The former cate'gory included language which was masculine or feminine but 
in the generic sense as well as specific references to individuals or 
groups. The M/F ratios for this generic categ9ry may be contrasted with 
ratios developed by dividing "average^ male references" by "aVerage female - 
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J- references" in tl^current study. ^ 

j ThesQ M/F , ratios are only approximates to the 'Tittle figures. " Because 
the present stifdy carried OL^t an item analysis separately for each gr:ade 
level, id'^A not incorporate. all items in thec^tati^Istical summary f or ^the ' 

'■'^ ' " ■ ■■' . ■ ' ■» 

25' ' ' . . 



'A 



\ 



ITBS Form Six tesults.^ How'ever, there is' substantial overlap In the 
material co^sidered by the two approaches, and since the number being 
compared , is a ratio, it 'is to a large- extent independent of the'jslze Qf 
sample of the items considered. Table 1 presents the comparison of the 
Tittle/fldta and thaf. of ^the present study, fn general,- ' the' differenc>es 
betwe an^lyses do nqt obsdure a cpnsi^erable comparability *^ts5ss t^sts . 
It is* unlikely that the components of the content' analysis u^ed in this 
' stud'y h^^ck£3torted.^the findings conx:erni|| content difficulty relatioi^ 
ships so greatly that they' cannot be considered compatible with the 
.earlier results of Tittle. ^ ' ' ' / ' ' * * 
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^ Quantifyi-ng^ Content in Related Material 



This ^tudy was concentrated on the item i^evel, -testing the "relation- 
ships between content and difficulty which were obtained ^t that level. 
However, the .quaAt^fication of content in order to. assess its impact^'on 
item stati^ics Is made^ difficult by the common practice. in testing of 
basing multiple choice items on some sort of associated stimulus material* 
This associated material can be readipg comprehension passages,, graphs . 
/ and tables, scientific diagrams, pictures etc. ' It introduces^a-rcompli-- * 
cation in the assessment of item^content, for thi6* content is^ associated' 



with the total set. of .Items "but^not pdrt of any given item. 



The .nature of the proirlem- is perhaps bewst seen by considering> a - * • 
hypothetical reading comprehension pas-sage which makes six male references 



and three^emale references and which is -the basis for four multiple 
choice questions. Clearly the content of ^ the"^p^ssage is relevant to the 
size pf the "differences ^etw^en the §exes which are founij'with respect 
to sudcess on these items. To show thi?' relevance, the ^passage contend 



Table' -1 



Broad Comparisoa'o'f H/F Ratios for 
• tittle Study and Current %tudy* 



T^st or Subtest 



STEP (Level 20) 



•Tittle 
Study 



Present 
Study- 



Rfeading 
Language AiKts 
Social Studi 
Mathematics 
Science 




3.90. 
57. 



^1 
IrS.OO 

2.50 

14.00 ' 



2.56 
5.42 
26.50 
1.88 



CAT -Mathematics 
(Level 5, Form 6) 



ITBS (Fbrm 6) 



Vocahullary and Reacfioig * 
_L3nguage Skills' . ' 

Mathematics ^ ')\ 
' Study Skills ^ ■ f ^ \ 

-4- , \ \ 

Metropolitan Achievement "Test \ / 

Primary II 




. ■ \ 
3.56! 



6.12 
. 1.78- 

2.12 
.1.57 



5.06 



The ratios for the present 'study consider most but not all of the 



items reflected in the ratio reported by Tittle 
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must be allocated to the items .in some way in order to carry out a 
sensible content-difficulty investigation/ 

One appjroach might be to consider the" nature of the individual 
questions and to attempt to* link paragraph^ con£ent to inj^ividual items • 
lor example, two of the male references and one of the female* referenced 
might be .attributed to a ceirtain item depending on the judgment by a 
suitable expert 'that this apportioning reflects passage televance to ^the 

« 

problems^ posed by the given items. Such a- model has qertain advantages 
in terms of a sensible allocation 9f passage content. If the questiori 
is, in fact, answered on the basis of tliose sentences containing, the ' 
specific content in question, then it seems- plausible, to consider that 
this specific content is the operative content in its impact upon item 
difficulty. A gr^at deficiency of such a method, of courae, would be 
the time requireld to make the expert judgments and the need to-^uard 
against overly subjeqjtii^e^r unreliable decisions. Almost certainly a 

n(mber.,pf judges wp^ula^be requil^ed. A possibly greater deficiency, 

- ^ H . - " * ' 

however, lie*§^ In ^its concept of t^ the mental process of the^test taker. 

The method is correct in asSimiing that content will effect item diffi- 

culty 4f».the test taker has- an information gathering process that reacts 

to this content. But it seems more likely .that the test taker's process 

is exploratoryl- and that it will range over the total passage. It seems 

unnatural, then, to allocate content only on the basis of its logical ' 

• * 

relevance to .items*. It seems more likely that the judgment of logical 

» , ^ * • . 

^ i^^fevance, made by a test specialist is a post facto decision, reached . 

s ' 

after reading the* passage 'and the question, and is not an index of the 
cojasfideration of • content that the test taker considers in the. process- 
of responding. * 
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Similarly, the likely process used by the test-taker makes it 
■I ^ •• '- 

unsatisfactory "tio average passage content over-the items. The (Juesbions ' 

. are based on the -passage. . The passage as ^ whole is considered in 
responding, eVen^ if the process of evaluation is one of rejecting, ' 
certain parts of it. The passage is nk . considered once, and then" the 
item decisions; -reached.- the .passage iT reconsidered in each decision. 
^'^^^^S^^tent, then, mtis.t be' considered to impact on each" item. 

Thl^ogic seepis\cons-istent with^the handling of - content in the 
variolas options of. an ifcl^t:<Option content is- summed to provide a 
.contribution to tcJtal item content. ■ It is not simply the content o'f \ . 
the correct response that . is. eva^ted, but the content of all. of the 
options, as a set. This -.practice, is clearly based on a model of test- 
taker process that sees the j^^spondent considering all of the^options- 
before responding, ' ' . • / * , *' . , 

Given this model,^;.the proper , technique for quantifying item 
content is to sum t^ie. content of all the relevant' components , of the , « 
item situation: passage, item st^m, 'item options. The impact of the' 
-content of ^stimulus-material Sudh as a reading passage is .thus,, in a 
serise multiplicative. The passage content is seen as .being considered 
by the tesfc-taker each time an item is 'considered. ' , ' " " ^ ' 

It is evident from this discussion that the impact of content on item 
difficulty is complexly affected i>y the model of the test taking process^" ' 
An examination of such Internal, process-oriented events, was beyond ' 
the gcope of tha present stuy, which analyses data gathered at prior / 
administrations. / Xhe as^umptidns aboufpr6ce3« which. have been made seem' 
appropriated. But the consideration of tests at the item level is only 
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beginning to receive appropriate attention. Studies* -'such as those' of 
Diirost and Hodges (1974) s^ejgi likely to expand our knowledge of process 
and ways of Mrroring if. Future studies of. the , test; 'taking process 
may well reveal ways in which, to improye the assumptions made here. 

Coder Reliability . , ' 

\ Complete double coding of all items was ryjt .fea$ibre. Double 

■'< •■•/.•'• r • • ' ' ■ . 

coding was carried ^out for 'selected items, for test^^ith significant 

content. These double codings were assessed using chi-square measures 

bt distribution agreetienrTJTB^ented .in Table 2. In general, th6 values 

of chi-square are sufficiently small to support *a high level of rater 

agreement. In an effort to place rater agreement on a scale more 

familiar to test users, the values, of chi-square* were supplemented by ^ 

»> , •> * 

an index, tt, proposed by Scott (1955) and defined ^as 

P 

€ 

1 - P 



TT = = 




where P^ = the observed* percent age of agreement and P^^ is the ^ 
expected by an hypothesis of chance overlap. It has a maximum value' of 
1.00 and a kno^n sampling distribution giving it a greater resemblance 
to interrater correl^ation coefficients. ^As with chi-square, it establishes 
that raters agree with' each other at a level o£ significantly 'greater 
> than chance. Results^ for this index are presented in Table 3.% -The leyel * 
of-kgreement among coders, ^S' suggested by the data in Tables 2 and 3 
was satisfactorily high. " . ' , ^ 



Chi-Squarfe Estimatfon jDf Reliability 
STEP Reading Form 2A 





Part I 


(Items 


1-30) 


ft 






/ 


Unique Entities 




.0.28 


(df = 


3) 


'P 


> 


.10 


References 


x^ = 


.025 


(df = 


^3) 


P 


> 


.10 

y 


Roles. • 


V = 


'■.713 


.(df ,= 


3)' 


P 


> 


.10 


Total Roles 


x2 


.0028 


(df = 


1) 


P 


> 


« 

.10 



Part. II (Items 1--3Q) 





Unique^ Entities 


x2 


= 1.754 


(df 


•= 4) 




. Referenced 


x2 


=. 1.212 


(df 


= 4) 


* 


Roles 


x2 


= .. 144 


(df 


1 5) 




Total Roles 


x2 


.312 


(df 


= 2) 




J' 


y 


Table 3 

4 







P' > .10 



Scott's' TT Reliabilities/ 
STEP Reading Form 2"a 

Part I (It^ 1-30) 

Unique Entities tt = .963, 

References ^ * tt = J 970 

'Roles - TT = , .7^9 

Total .Roles tt .= ^ .-^8701 

Status " TT = "1.000 

I , . , 

j 

■ ?art II (Uems l:-.30) 

Unique -Entities tt ^- ^853- 
References tt = ^ .942 

Roles TT = .912 

* . *. 

Total Roles^ tt = .882- 

Status ' -TT = .825 



] Results 

^ Table 4 J)resents descriptive inforfaatipn concerning the tests and 

samples: the'number of ;|tteins, and^comparative data, by sex, on the sampl 
i - * - y . 

size, the means, the standard deYiations and the intemali^consistency^ 

reliability estimates. In all, there were 36 subtests identified, eaclf 

of whieh was ite^ analyzed for males and 'for females. The minimum' sample 

size used was 1,000 cases; the, maximum was 3,384.\ 

Sample size, of|jc.ourse, is an important factor in fhe .determination 

of statistically meaningful differences. For = - 1,000, and for 

•an average proportion in the two groups of 0.50, a difference in propor- 

tions of approximately- .06 is significant- at the .01 level. For the " 

larger samples in the stydy, and for values of the average proportion 

other than 0.30, the minimum significant difference is* even smaller. 

Thus,-^ the (Sample^ sizes included f this study of fer* a sensitive 

^ . ' . * ' ' , * ' * « > 

statistical test of differences between the sexes* *f \ 

« The score %eans for the varlpiis sub t^a^ generally, show equivalence 

between the sexes, with little evidetrce'of superiority 3or one sex or 

the other. While many .of these, score differences achieve statistical 

Significance,, because of the lafge* sampled involved^ their^ practical 

^significance seems, limited.' 

A' number of. the sex contrasts are in the iiirections suggested by- 

stereotyping. Thus, significantly better .perfomMnces are exhibited - 

by JEemales 'on both sections of the Metropolitan Achievement Tests: 

Reading and on the Iowa Test of Baste Skills Reading tests at Level. 11 

CGrade.5) and ^vel 14 (Grades). However, while females score consi^ 

^entiy higher di- the vocabulary tests for ITBS - Level 11, ITBS - L^^ST 

14 and STEP 2A, the differences Wre. not statistically significant, and- 



, ^ 4 • 

Table 4 ' 
. Characteristics of tjhe Item Analysis Sampl es 



Number 

of ^ . Stafidard 

t Tatfle ' Items ' Sex^ N Mean Deviation 




Grade 10' Tests 



1 



\StEf>j2A Reading ' * . 1, 129 22.1 5.0 

Part I 30 M 1,070 /22.1 5^3^ 

■ / ■ - . 

STEP 2A -Reading . ^ F 1,129 12.4 6.0 

Part II 30 ^' M 1,070-. 12.2 ' 5.8 

STEP 2A MeoAanics of F ' 1,320 29.0 8.3 

■ ^ ■ Writing, Part I 45 M= 1,130 25.1 ' 8.7 

>•,'.. 

STEP 2A Mechanics of F 1,320 . 16.0 13.7 

Wrp.tin^,\part II' 45 . M 1,130 15.0 12.5 

STEP 2A English ' F- 1,329 22.6 *^ 6.4 

--Expression .40 M 1,168 "21.5. 6.6 

^Jt*.- * ' I A* 

"^XEP, 2A English ^ " 1,329 10.3 5.7 ^ 

Expression 25 M'7 l,168r • 9.5' . , 5.2 

, STEP 2A Basic Math , F 1,303 24.4 7..^;- 

r " Concepts ' 50 M ,1,038- 25.3 ' . 8;t^^.. 

STEP 2A' Social Studiea- F 1,26Q 19!4 6.4 "i^' 

Pdrt I . 35 M 1,110 20.6 '■'6.9 

- \ 

STEP 2A Social Studies ^ F' 1,260 13^.1. 6^6^^ 

Part II -■ 35' , M . 1,110 - 14.3 '7;2^ 

STEP 2A Science '. • e , |. 1,248 2*3.2 6.7 

Part I • , ' ,45 M 1,237 .25.4 ' 7.6 

^ STEP-2A Science , 'f 1,248 11.5 6.9 

• . Part 'il . . ., 30 ' M 1,237 11.'6 _ 7.4 
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■' Tablfe 4 (cont'd) 



Characteristics of ,the Item Analysis Samples 



Test Table 



Grade 8 Tests/ 



ITBS Vocabulary 



ITBS Reading 



Ntimber 
of' 
Items 



48/ 
80 



Sex 



F 
M 

F 

'm 



N 



• Standard 
Mean Deviation 



IjOOO 38.6 8.3 

1",000' 28.1 8.4 

V / 

1,000 49:5 ' 13.4 

1,000 .-47.7 14.5 



ITBS Spelling 



48 



F 1,000 
M- - 1,000' 



27.7 
22.6 



9.4 
9.6 



ITBS Capitalisation 



44 



F , .1,000 
M4fc.000 



29.5 
25.4 



7.3 
•8.1 



ITBS Punctuation 



44 



1^1,000 
M ';»'"i^00d 



26.16 
22.2 



7.9 

7.-9 



ITBS Usage 



ITBS Maps 



32' 



42 



F 
M" 

F 
M 



1,-^OQ 

i-,doo 



1,000- 
1,000' 



17.0 
14.6 



4.1 
25.4 



6.1 
5.7^ 

7.3 
8.0 



ITBS Graphs . 

.ITBS Reference 
Materials 



28 



59 



F 1,000 
M * 1,000 



F 

'M 



1,000 
1,000 



17'. 0 ^ 
17.2 ' 

37.5" 
33.7 



5;1 
5.5 

,9.6 
11.0 



ITBS Math Concepts 
ITBS Math Problems 



1 

'48 



34 



F 1,000 
M "1,000 



2 8. -2 
28.0 

17.1 
17.5 



8.9 
9.2 

5.0 
5.7 



r 



T?d3le '4 (cont'd) 
Characteristics of the Item Analysis Sajnjrles 



Tes 



t Tables ' \ 



Grade's Tests 
" ITBS Vocabulary 

ITBS ' Reading • 

ITB^ Spelling 

ITBS Capitalization 

ms Punctuation 

k 

ITBS Usage 
ITBS' Maps^ 

ITBS Graphs 

ITBS Reference 
.Material 

ITBS Math Concepts ^ 

ITBS Math Problems 

r. ^ 



Number 
Items* Sex 



43 



74 



43 



40 



40 



32 



36 



* . ^ Standard 
N ■ ■ Mean Deviation 



M 

F 

"f 

M 

F 
M 

F 
M 

F 
■M 

F 
M 



1,000 26.2' J " 8.-0 

l',006 25.7 , 8.7 

1,00&' 47.0 • 11.1' 

1,000 -AS^O .^^13.2 



'1,000 

.1,000 
1,000 



' 1,000 
1,000 

1,000 
1,000 



26.7 
23. « 

26.6 
24.0 

28.0. 
24.8 

19 ". 3 
17.2 



8.0 

7.2 
7.9 

■7'. 1 • 
8.0; 

5.9* 
6.^ 



1,000 22.6" .0 5».8 . 
1,000 .23.2 ^'^-^f 

1,000 17.3 ' , .4^^ 



26 


° -M 


' -i-,ooo 


17.3- 


•' ' 5.5 

. * 


56 _ 


. F 


•1,000 
,^i)0 


36.0 
32.8 


10.3 
. 11.5 


42 




"''1 ,7)00 

1,000 


25.8 
■26.2 


. 7.7 
■8.5 


29 


M 


\f,000 
\l,000 


.1-7.0 
■16.4 


• 5.2 



T^ble (contVd) 



^' - Charact-e&tic3 -of the Item ^aly^is Samples , 

/ ' ^^2pf./ 'V • Standard 

/ Test Table ^ It' ems ^ Sex. f_N_ Mean - '^ Deviation 

Grade 2 Tests . , ^ . v 



Metropolitan Achievement * ? ' ^ - \ ^ - ^- . ^ . 

Tests, Ppimary II \ ' i ; ^ F - ^ 3,265 ' 9.8 3/2, 
'Sentences * - 13 : M 3,384 ^ jB.9 ^ 3.4 



7 



Metropolitan Achievement . * 
Tests, Primary It , F 3,265 22 .'5 - 7.8 

Stories ' ^ » 31 IT m' 3^384 20.0 S.^Zt 



this'-is also true for the Passages of STEP 2A Reading Part II.. Thus, 
the female' superiority on verbal material Is centered more on reading 
comprehension than on vocabulary,, and tends to^bermore propiinent in the 

lower grades. * ! ' " . » ^ * 

$ • ■ , 

The more formal aspectTs of language usage: spelling, mechanics of 

writing^ effectiveness of expression, etc., all favored f emales ,1' with 

^ - *f* ^ ' 

some of the largest intersex dif f erences^occurring in these areas. 

t ' ' ' ' ' 

For Mathematics, the pattern shows a statistically significant 
difference (p <^.01) in favor of Grade Ifl males on STEP Mathematics 
'2A; a statistically significant difference (p < .05) in favor of Grade 
5 females on ITBS - Level 11 Mathemapics Problems; but no other signify 
icant differences. These patterns of performance oii mathematic?^ tests 
are in accord with other studies which show that significant sex \ 
differences in favor of males do not appear unt;il the high school years. 
On the test score level, then, the evidence of se# differences presented 
by this relatively large array of instruments held few surprises." The 

general effect of content on scores seemfe^* to be about/as described in 

' '\ ■ ' ' ^ - , ' 

earlieg: researches. The pattern seems to be (a) female ^guperiprity on 

verbal tasks throughout the elementary years, but no longer extending • 

finto the secondary leviel, (b) sex parity on arithmetic/mathematics in 

[ ^ ^ 

the elementary 'y^ars, but a superiority for males during the ^secondary 

- ' * ■ r ' ' 

years. • ^ 

Item Chajaoteristics Summary - . 

^ The present |nalysis was focused on the item level, "rather than 
the test' leyel. As a result of th§, detailed analysis, a number of 
descriptive characteristics were available for each it6m. These were 



logically divided into two classes. The first claiss ii^cluded descriptions 
of the statistical characteristics of thelLtems, such as the percentage* 
of males or females succeeding, the difference in these percentages, etc. ' 
These descriptions were essentially the type of information whic^i is ^ ' 
referred to as item analysis in test construction. .The second 'j:l'as^ " ' 
included descriptions of the content characteristics of tiie items: the 
number of referencesfto males,* ta females, to "neutrals^' (people of * * ' 
unspecified sex), etc. The content analysis was,_ of course,- an "item 
analysis" also; only convention reserves that label for a ^^scription 
of the statistical putcomes of administering the item to a sample. 

The statistical item analysis included item-total correlations as 

« - 
well as the distributions of subjects across the possible responses. "For 

• , tj/x * 

these item-total correlatio^^ the "total" score was defined by the patterns 
o f ^a j ^ mj^ Tls t r a t ion an4 was: based on those items administered together 
in a separately* timed session. 'Thiis, STEP Reading has, in all,' 60 items, ^ 
but the statistical analysis wag performed separately for each 30 item ^ % 
section. This approach was a logically necessary step in considering tjie 

proj^lem of drbpout, or failure to^^omplete, which is essentially defined 

. ^ . ^, 

only for a given, timed sessioti. ^ ^ . : 

♦ . , ■ / • ,0 

The level of generality of the test content used to define homogeneous 
' "I * ^ \ 

item-'Sets is therefore an important decision. The fundamental ,investigatipxi 

.cenliers* on a measure of association between statistical and content chkrac?^ 

t^risticsy .An association tested across ite^. Demonstrating an association 

for limited subsamples of items for reading passages, or for mathenytlcs 

problems, 'Was not a principal goal of the studyT In a study involving many ^ 

hundreds of items, the potential for such subdivision seemed endless. 



Item material in tests is presented in a variety of manners and formats, 

» , a ' » . 

all with a potential for .interaction witTi sex differences and content ' 
fa.ctor^. However; such very small sets of items* do not Constitute a 
large , enough paifiple to do more^than suggest relationships. The present 
study, therefore, sought to define meaningful groups of items which were 
sufficiently homogeneous in content to constitute a category, but did^ ' 
not seek t© consider each .of the, many specialized item sdbgroups^hich ' 
could be defined. The consequence of this is, of course,, that negative 
findings carrot mean that there are no statistical-content r^afa^ionships 
but that at the leVel'^of analysis Considered here, none were found. 

Table 5 presents descriptions of the fourteen tests that weire studied. 
For each set. of items, the means and standard deviations of the item ) 
descriptors are presented. These descriptors are divided into the seVgh 
statistical characteristics and ten'' content factors. 
\The five statistical characteristics are: 

1. Percent males passing, including^Omits bijit exclujjing Not Reached. 
Thus, p'exceat passing is defined as the Total' Right divided 

by the Tot^^l Right plus th^otal ' Wrong plus .the Total 
Omits. 

2. Percent females passifrfg 

3. The difference in the j^ercent passing, 4^fined as MalesrFemales. 
■^'4. Male item^tptal biserial. This coefficient iexcludes the item 

7 itself from the total, , ! * - 

5. Female item-total biserial. ) ' • ■ . - 

. *The ten content characteristics were: 

1. Total tnale references ' . * ^ 

2. Total, female references \ • . ^ 
3.. Total neutral references! 



4. The^ difference between male and female references ^ 
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TABLE 5 . 
« 

Means and Standard Deviations for. - 
Selected Statistical and Coustent Characteristics 

^ Statistical Characteristics 
STEP 2A 





X 

Male 

Pa;88lng« 


t -2 
Female 
Passing 


Sex. 
Difference 
•» in Passing 


Male Item 
Total Bis€ 


M 


.53 






^ .« 


"'sn 


.22 . 


• 22 


.OA 


.18 ,= 


M • 


.49 


.54 




.56 


sn 


.17 


.19 


.05 


••.21;,. 


* M . 


.48 


.45 ' 


.oa ' 


.48 


SD ; 


.16 


.16 


.04 


.16 


H 


.S> 




.03 


. ..42 


*SD 


.19 


• .21- 




\l2 


M 


.49 


.46 


.04 


■ . Zso 




.16 


.17 


■ .06^ 


.19 



Female Item 
Total BJserial 



Reading 



Language 
Arts 

Social 
' Studies ' 

Math^iAatics 



Science 



•49 
.17r 
.59 
.23 
*44 
.16 
.38 
.14 
.44 
.21 



Contefi^t Characteristics 
STfeP 2A 



T" 



4r 



Total Total Total 



Male- 



TotaX Total Total 



Riiading 



SD 
M 
SD 
M, 
SD 

Mathe^tics M 
SD 

Sdience M 
SD 



« Language 
Artk 

'Social 
Studies 



17.48 
21.89 
. .65 
1.39 
1.06^ 
1.57 
.32 
1.X7 

. .20 

'.89 



Total Total Total 
Female Neutral Female .. Male Female Neutral Male Female Neutral 
'Ref , ^ Ref . Ref. Actors Actors. Actors Role^ Roles iBles > 



^6.83 
,19.35 

.12 
' /.66 

.04 

. .17 

.'■4J 

. .00 ' 



44.03 

I5i^75 ■■ 
• .74 
1.17 
8<36 
9. .45 
.45 
1.6f 
' .45 
'.89 



10.64 
24.iS' 
.53 
1.56. 
1.01 
1.57 
. .15 
1.47. 
• .20^ 
.89 ■ 



3.i5 
3.19 
.33 
.70 
.59 
•1.00 
..15 
.59. 
.09 




40 
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TABLE 5 (contM)^ 



Means and Standard Deviations for , 
Selected Statistical and Content Characteristics 

Statistical Characteristics 
ITBS - 5 ^ 







A 

Male 
Passing 


% 

Female 
Passing 


« ' Sex 

Difference / 
in Passing 


T ■ — 

• Male Item ^ »^ Female Item 
Total Biserial''' Total Blserlal 


.Vocabulary 




• 60 


.61 


-.01 


.49 ' 




.43 




& Reading 


SD 


.16 


.18 


.05 






.^-3 




Language 




.55 


.63. 


-.07 


.47 




.46 . 




Skills 


SD 


.14 
















.15 


.04 


.14 




.14 




Mathematics^ 


M • 


.58 


.58 


.00 ^ 


/50 




.44 




\ 


SD 


.16 


.17 


.^5 


.12- 




.12 




Study • J 




.62 


.64 


-.02 ' 


^ \48 




.43 




Skills \| 




.12 














SD 


.06 

* 


.11 




.12 








t 

i 


Content Characterist 


ics 
















XTBS - 5 


I 








' — 


Total ^ Total ' 
Male Female 
Ref. Ref. * 


Total. 
Neutral 
'Ref. 


Male- V Total 
Female Male 
Ref. Actors 


Tk>lal Total 
Female Neutral 
Actors Actofs 


Total 
- Male 
Roles 


Total 

Female 

Rol§s 


Total 

Neutra^" ^ 
Roles 



Vocabulary 'm 
& Reading^ 

Language M 
Mathematics -M 



•Study 
Skills 



M 
SD 



11. 1& 
17.12 
1.07 
.1.66 
.32 
.92 
1.03 
<3.96 



. 3.40 
10.76* 
.99 
1.93 
.35^ 
1.34 

2.87 



2.50 
3.03 
1.32- 
'1.63 
.37 
1.5? 
2.03 
2.86 




1.23 
1.73 
.32 
.74 
.00 
.00 
.41 
.82 



.35 
.75 

i;o4 

.08 

'•33^ 

.48 



.78- 
1.29 
.19- 
.97 
.11 
.36 
.70 
1.26 





TABLE 5 (cont'd) 

4 V Means ind Standard Deviations for , 
Selected Statistical tind Content Characteristics 

Statistical Characteristics , 
/ / ^ ITJJS - 8 • 



• 




X 

Male 
Passing 


Female * 
Passing 


Sex 
Difference 
in Passing 


Male Item 
Total Biserial 


Female Item 
Total Biserial 


• 


Vocabulary 


M 


.58 


•60 


-.02 




.46 




.44 






Reading 


•SD 

* 


.16 


.17 


. .06 




.15 




.15 






Language 


m/ ^ 


^, .48 . 


.58 






.44 




.45 






Skills 


SD 




















. .17 


.05* , 




.13 




.14 






Hatheoatics 
1 


M 




.53 


.00 




.42 




.39 








SD 


.15 


.17 


.05 




• 13 




.17 






Study 


. M 


' .58 


^ .59 


-.02 




.45 




.41 






Skills 


SD 


.14 

/ 


.15 


.06 , 








.13 






V. ■ -• 






Con):ent Characteristics 




















ITBS - 8 
















Total . Total 
Male, Female 
Ref.' Ref.' 


Total 

Neutral 

Ref. 


Male- Total 
Female Male i 
Ref. Actors 


Total 

Female 

Actors 


Total 

Neutral. 

Actors 


Total 

Male 

Roles 


Tfatal Total 
Female Neutral 
Roles - Roles 





Vocabulary 
j& Reading 

Language 
SHills 



M , 13.19 
SD - 24.46 
M '1.76 



SD 

Mathematics M' 
SD 

- Study M • 

Skills • 

SD 



2.95 
;57 

1.21 
^36 
.87 



\58 
1.81 
.60 
1*66 
.07 
.38 
.26 
.83 



8.38 
9.2? 
1.40 
1.79 
.20 
.81 
2.19 
4.58 



12.61 
24.61 
1.15 
3.51 
.50. 
X.22 
.11 

r.2i 



2.59 
4.29 
.86 
1.15 
•24 
.51 
.30 
.74 



.20 
.62 
.34 
.84 
.05 
.27 
.17 
.53 



3.27 
3.83 
1.22 
1.35 » ^ 

.07 

.31 
1.42 
2.18 



1.17 
1.51 
.31 
. .72 
.01 
.11 
.10 
.30 



.10 
.33 
.24 
.73 
.01 
.11 
.02 
.20 



2.01- 
2.51 
.36 
.73 
.10 
.34 
.10 
..3p 



> 



42 



ERIC 
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^ TABLE.5' (contld) 

Means and Standard Deviations for ^ 
Selected Statistical, and Content Characteristics 

Statistical ^Characteristics - 
Metropolitan Achievement Test 



Male ' 
Passing 



Female 
Passing 



Sex N ^ ' ■ 

Difference 'Male Item^ . Female Item' 

in Passing ' Total Biserial Total Biserial 



•Reading 



M 
SD 



.63 

".09 



.70 
."10 



-.07 
.03 



.64- 
.11 



• 68 

\l5 



Content Characteristics 
Metropolitan Achievement Test 





Total 
s Male 
Ref. 


Tofal 
Female 
Ref. ' 


Total ^ 
Neutral 
Ref. 


Male- 
Female 
Ref. 


Total 

Male 

Actors 


Total 

Female 

Actbrs 


Total . 

Neutral 

Actors 


' Total 
Male 
Roies 


Total 

Female^ 

Roles 


Total 

Neuttal 

Roles 


Reading 


M 4.50 


.89 


2.90 


3.61. 


1.86 


' .70 


1,68 " 


.1.00 


.52 


• 32 




^ SIK 5.00 


1.38 


3.77 


5.72 


1.95 


1.02 • 


-2.OL9 


J..31 


' .9Q 


w71 






































I 






I 
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I TotaT female a(^tors t • * 

\7'« Total neu,tral actors ' ' - * 

8. ^, Total male roles ' ' - 

9. Tota l fe male roles t ^ 
-10. Total neutral roles 

The averages for th?e item statistical characteristics reflect the 
same fundamental test qualities of difficulty level and reliability which 
are descri1>ed in Table 4. Accordingly, no special dia,cussion of them is 
necessary. ~ ' ^ ^ ^ * ' 

;The .averages for th'e content characteristics reflect some interesting 

patterns. These may be summarized as follows: 

»• 

N 

1. References to people are predJJifQinantly found in reading and 



• * vocabulary tests. There is a marked reduction in people 

n • - ^ - 0 

- » references in mathematics '-or science. * 

2. - Within the reading tests , references to males ar^ about three 

• ' times as numerous as references to females for Grade 10 and 

Grade 5 material, about 'fiye times as numerous for Grade 2, 
-and about twenty times as numerous for Grade 8. Only once in 
fourteen tests did female references exceed male references,^ 
and. this was in ITBS - Mathematics,^ Grade 5.. 

3. Neutral references in general show higher ^frequencies than C 
, females* Only in one test, .ITBS - Vocabulary and pleading, 

\ 

Grade 5, did female references exceed neutral references. 
Neutral references exceeded male references on seven of the 
tes-t^s, tut were exceeded by male references on seven others. 

4. The numbers of different actors return these esS'entlal patterns. 



'I 

9\ 



Female actors are much less frequently mentioned than malels. 



44 



The totals of neutral actors exceed those of either males or 
females in ten of the tests, suggesting that the rS^ferences- 
to-actors ratio^fox male references may be somewhat greater, 
since neutral references Exceed male references In only ' 



seven of the tests/ 



The ideatificati^n of roles- is most difficult' for any but the 
reading tests. This suggests that the descriptions of -people^- 
in these other tests are very limited. laj^ihe four ^.reading t't^'A 



sets there are trends tjQward more role^Wp^ item in the tests 
for the upper grade levels. Male and finale roles .show a dip 
at .Grade 8, but neutral roles ^lfe|f3&rease systematically. 
Further, the ratio of ^^^l^^^^^^^^ increase is such that while 
they' constitute only about 20% of roleg at Grade 2 they consti-^ 
tut'e about 65% of ^roles ^t Grade 10. It seems appropriate to i 
characterize these trends as more roles and more complex roles, 
for the higher grades » 



Correlational Analysis ' ' ' ; 

- The selection of characteristics results in a 7 5^10 correlatipfira; 

' • ' * ^ • T 

matrix: for each of the 14 achievement test s.ections. The faults of 

these analyses are presented in Table^6 through 19. " ' ^ 

• / 
The most important s'tati3tical characteristic is the sex difference 

in percent passing. To what extent is this associated with content 

facttwrs? Tables 6 through 19: prdyjLde 140 .correlation coefficients of 

* - ) " 

thifii Variable with content. Of these, 28 or 20% are significant at 'the 

.05 level or greater under assumptions of biVariate normal di.strihu£ion. 

This is xiOt'^A of 140 independent relationships.' In viey of the 
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TABLE 6 



\/ent 



Intercorrelations of Selected Content Factors and 
Item Characteristics, Grade 1^0^ ' ' 



/STEP -Reading, Form 2A 



N =,^60 Items 



% ^ . % Sex ^ ^ 

Male Pemale Difference Male Item. Female Item 

Passing Passing 'in Passing Total Bigerlal .Total Biserial 



Total Male 

References -.53** -.48** -.26** 

Total Female 

References j * -.27-* -.22* -.27* 

Total Neutral . <^ 

Refer.ences -.-64** -.6^** -.16 ^ 

Male Refer- 
ences Minus- ^ , * 
Femal^ Ref ,-.25* -.25* . -.02' 

Total M^e . * 

.Actors- ' -.59** -*56^* -.18 ^ 

Total Female J j . ' 

Actors „ .-.32**' -.26* -.32** 

'Total Neutral • - 

^Aptors -.51** -.48**. -.17. 

To^al Male 

Roles ^ • -.47** - -.43** 7.25* 

. Total Female 

Rores ^ -.32** -.26* -.32**, 

Total Neutral ' . 

Roles ^ ' * -.49** -.47,** -.14 



-.45** 
-.18 

-.37**' 

-.25*. 
' -.36** 

-.27* 
'-.20 

-.35** 

-.26* ^ . 

I 

-.29* 0, 



-.29* 




-.24* 

-.08 

-.07 

-.20 

-.07 

-.22 



*' a.p < .05 

•** - p < ^.01 



ERIC . 
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* / TABLE 7 

Intercprrelations of Selected Content Factors and 
^. -Item Characteristics, Grade 10 



STEP ^Language Arts, Fotm 2A 



Sex 



N,= 155 Iten^ ^ Male \ Female , Dif^rerice Male item . Female Item. 
^ H Passing Passing in PassiTig Total Biserial ^otal 'Biserial 



Total Male * 












References . 


-.03 




■.05^ 




.of 


*/TotaX'Female' 








f 




References 


-.04 




.02 


* 


-.06 




■ 










References 


.or - 




.06 




-.28** 


* Male Refer-" 












ences Minus 












Female Ref • 


-.02 








/.08 


T^otal Male 
Actors 






.0^ 




.01 














Total Female 












Actors 


-.06; 




.^4 




-.04 


\ Total Neutral > 










I 


Actors 


-.11 _ 




.17 




--.27**, 














Total Mate 




■ / 








. Roles . 


.03 ■ 


' ( 


.'02 




.04.^ 










/" 




Total Female 












Roles 


-.01 

9 




,04 






Total Neutral 










^'./^^ 


Roles 


^ .01 


' -,. ,--"2 , 

4 

1 




Ml * 


* p < ;.05 

p < ,01 \ 

. i 




** 

4 









.00 



.17* 



-.07 



-.08 



.11 



.05 



.18* . 



-.06. 



-.03 



.11^ 



.14 



/ 



-.05- 
. .03 



.15* 



jrOl 



.17* 



.--.02 



JD3 



ERIC 
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TABLE 8 



Intercorrelations of Selec\:ed Contejnt Factors and 
. * I^em Characteristics, Grad^ 10 



STEP Mathematics, Form 2A 



% . %' , Sex - ■ p 

N = 65 Items Male . female Difference ifile Item Female ttem 

" Passing Passinjg. foi Passing Total Biserial Total Biserial 



Total Male 
References 

Total Female 
References 

Total Neutral 
References . ^#03 

Male Refer- ^ 
ence Mintts ' ^ 
^ Female Ref • , .IS^* 

. Total Male 
Actors .20 

^ / - f 

Total Fefliale 

Actors ' * -,,08* - 

f 

Total l|eutralf 
- Actors , . .01 

' Total Male' r'^ 

I "^Roles. " ,12 

Total Female 

Roles - -.15 

Total Neutral 

Roles. ' .10 



.06 



-".10 



-.02' 

.11 
..13 

-.04 



-.03 



-.15 
.10 



.11 
.16 



.19 
.13 
.16 " 
.27* 
.07 



-.oi" 



' .16 
.06 
-.06 



.11 ^tr*" .09 



.20 
.07 



'.01 



.15 
-.07 
-.05 



.16 



.12 



-.01 



.06 



.21* 



.12 



.05 



.14 



-.10 



-.05 



* A p < .05 
** = p< .01 



ERIC 
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.TABLE 9 



Intercorrelations of Selected Content Factors' and 
Item Characteristics,' Grade 10 



■J „ 



STEP Social Studies, J.eveX 2A 



N = 70 Items 



• % ^' %. Se^T I 

Male Female . Difference Male It6m Female Item 

Passing Passing in Passln^g To]j;al Biserial Total ^Biserial 



Total Male 
References 

Total Female . 
References 

Total Neutral 
References 

I Male Refer- 
' ences ^Minus 
^ Female Ref . 

Total Male 
Actors 

Total- Female 
' Actors 



Total Neutral, 
Actors * 



Total Male ^ 
Roles 

Total Female 
'Roles 

<v.Total Neutral 
^ Roles 



.00 
-.05 
-.05 

.00^ 
.01 
.04 
.02 

-.03 

-.05 

-.14 



* =» p < .05 
** = p < .01 



.04' 
-.04 
-.07 

.05 
.08 

.06 .- 
-.01 
-•.03 
-.04 
-.15 



-.15 



-.01 



.11 



-;.15 



-.27* 

-.09 

-.09 

.04 
-*02 

.06 



-.05 
-.11 



-.07 

-.03* 

-.14 

-.12 

-.07- 

-.09 

.-.i'l 

-.03 



-.02 



-.14 



-.09 



.00 



-.13 



.11 



-.07 



-.12 



-.14- 



-.04 



r 
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: TABLE 10 



. Intercorrelations of Selected Content Factors ^ind 

* * ' Item Chara.cteristics, Grade 10 ^ ' " 

* 4 

' ^ STEP Science, Level 2 A' , " 

' ~^ %^ Sex ^ yn^" 
N = 75 Items Male * Femalfe ^Difference Male Item Fetaatle Item 
y Passing Passing in Passing Total Biserial Tot^l'Biserial 



Total Male 
References* 

Total Female ; 
References 

. Total Neutral 
References* 

Mal^Refer- 

ences Minus 

. Female Ref • 
t 



.20* 



\ N/A 
. -.01 



.15 
N/A 
.00 



^10 
N/A 
-,03 



.20* 

It. 

.12 



Total Male V 
Actc^ , . ^ 

Total -Feaali?' t 
Actoip ' \ ' N/A 

... °. 
^-.10 




.13 

■ oOJL «». 

,.13 
.27** 

N/A 



.16 

N/A 




.1(5 
.28*^ 

. N/A 



Total, ifeu^r^ 
Actors^ ' * 



Total Male ' 
Roles 

* Total Female 
Roles 

Total Neutral 

Roles 

f ■ . 



-.05 



N/A' 
■/.12 



-.01 ^ 



N/1 



-.24*. 

^26*'" 
. N/A 



.09 
.20* 



A 



.-.07 ^ ^ -.13 



N/A 
.00 



.05 
.22* 
N/A ■ 
.0^ 



* = P < .05 
** = p < .01 



ERIC 
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TABLE 11 




Intercorrelations of Selected Content Factors and 
^ "Item Characteristics, Grade, 8 

ITB^, Vocabulary and Re.ad^g, Level 14 




= 128 Items Male 



Total Male 
References ' " 

Total Female 
References 

Total Neutral 
References 

Male Refer- ' 
ences Minus 
Female Ref . 

Total Male 
Actors 

Total Female 
Actors 



,25** 



-.14 



.27** 
• 13 
,23** 



Total Neutra£|^ 
Actors 



Total Male' 
Roles 

To'tal Female 
Roles 

Total Neutral. 
Roles 



-.06 



.15*' 
/ 

,23** 
,17* 



% Sex 
Female Difference 



Female Item 



Passing Passing in Passing Total Biserial Total Biserial 



..28** 



-.23** : -.20* 



-.09 



.11 " 



-.06 



-.13 



.29** -.11' 



.20* 



-.20* 



•=:.02 



.16* 



-.20* 



,22** 



-.21** 



-.06. 



-.13 



:o5 



-.05 ■; 
^.17* 



r.09 
-.12 
.08 

-.08 
.00 

-.12 
" .12 • 
.01 

-.12 

' .19* 



-.10 



-.14 



.03 



-.09 



.01 



-.14 



.07 



-.05 



-.14' 



.15* 



* =, p < .05 
** = p<" .01 
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jfW . , X ' TABLE. 12 ^ ' ' ! 

' \JJntercorrelations of Sel^ected Content , Factors and 
. 7^^'"' ' ^ ' Item Characteristics, Gi^4q^ 8 




\ 



IT^S, Languagev Skills Level 14 



%• 
^e 



% , Sex ^ : , - . \ 
/^"N = 1^68 Items ftafle , Female Difference Male Item Female Item \ 
Passing Passing in Passing Total Biserial Total Bisejrial 



Total Male- 

iReferences .01 

Total Female ^ 
References * -•02 

Tot£tl Nieutral 
References .05 

Male Refer- 
ences Minus 

Female Ref. ' -.02 

Total Male 

Actor^ ' - ^'^^ 

TSiXal F^le 
*"i?otors' -.01 

, To~tiaJ'lieutral 
'"^Actors , . .01 

-r 

Total Male 

Roles -.03 



-.02 
.07 

.01 



.01 



c 



Total Female 
Roles . 



• 04 



^.01 
.02 
-.03 
.03 



.02 

i)l 
-.11 



•01 
.08 

.00 



-.02 



-^.02 



.01' - 



-.i3 
-.09 ., 
.05 • 

-.07 ' 
.21** 
-.08 
-.03 
T-.08 



-.07 



-.11 



-.03 



.06 



-.08 



-.15- 



-:o3 



.00 



-.06 




^ . -.09 



Total Neutral 
Roles 


' T 

.02 -.01 ' . 


.0^" 




.12 • 




- -.16* 

1 




' • *. ' 


1- 







* = p< .05 
** = p <^ .01 
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TABLE 13 



Intercorrelatlons of ^Selected Content Factors and 
Item Characteristics Qrade 8 ' 



ITBS, Mathematics, *Level 14 





% 


■ % 


Sex 






ML<= 82 itejas- 


Male 


Female 


Difference 


Male Item 


Female It.em 




' l^assing 


\J o o O "f T> Ct 


xn Jrassxng 


lotax Blserxal 


Total Biseriai 


Total Mle 












Referepfces 


-:o2 , 


• 


"".UX / 

t 


— n/i ' 


-.06 


Tot^ Female 












^Jkfxerences - 


. .14 — 


HQ 




. lU 


.12 


Total Neutral , 




f 


t? 






Ref erehcelf^^' 


-.15 






— 1 0 

— . IZ 


. -.16 


Male Refers. ° 










ences ^linus 






— X 






Female Ref. <^ 






-.05 


-»oz 


.10 




Total Male 








V 

-.07 , 




Actors 




-.11 


.00 


■ ' -.08 




Total Female 




t 










Actors. 
« 


.16 


^' .10 ' 


•16 


.15, 


.16 




Total Neutral 








< 




Actors 


-.18 


• -.18 V 


^ .05 , 


-.15 ; 


-.17 • 


Tptal Male 












Roles 


-.10 


-.12 ' 


f '.08 


\ .01 




Total Female 












Roles 


.14 


. .09 


.14 


_ .19* ^ 


■ .19* 


Total Neutral', 












Roles 


^.22** 


^.21 


. .03 












\ 





* 



p< .05 
.p< .01 
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^ lABLE 14 "• 



Intercorrelations of Selected Content Factors and 
Item Characteristics, Gtade 8 

ITBS, Study Skills, Level -14 



r>% % . Sex 

N = 129 Items " MaL 



• Male' *Female Difference Male Item / Female Item 
Passing ^ -Parsing' ^ligi Passing' Total Biserial Total Biserial 



Total Mafle 

References ^ .07 

Total Femaje 
. References ^12 

Total Neu4:ral 
References -.12 

Male 'Refer- J 
ences MiffaS^ ^ - 
; Female/Kef. ' -.01 

Total Male 
-Actors / .04 

Total Female 

Actors .13 

Total Neutral ^ 

Actors -.15* 

Total Male 

, Roles. - -.14 

Total Female 

Roles -.03 

Total Neutral 

Roles -.<)7 



.09 : 
-.06 



•.OX V 
.06 

-.03 . 



-.OA 
'.04 
-.13 



-.06. 



-.07 
.01 
-.24* 



'-.11 -^.03 



' -.04 



• .00 



--.03 



-.15* 



-.qi 

.06 . 
-.29** 



•-.b5 



V , -.04 
^ ,07 
-.26** 
/-.31** 
. -.06 
-.20* 



-.01 
.04 
-.2*5"** 



-.04 



-.03 



.07 



-.11* - 



r.28** 

-.09 

-.17* 



* = p< .05 
** = .01 



r 
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TABLE 15 • 



Intercorrelations of Selected Content Factors and 
Item Characteristics, Grade 5 



ITBS, Vocabulary and Reading, Level -11 . 



% 



Sex, 



N « 117 Items MaU ' Female Difference jtelfe Item 



Female Iten^ 



PassinR.- . Passing jn "Passing Total Biserial Total Biserial 



Total Male 

References ' -.20* 



Total .Femal^ '-^' ^ 
References ^ : .00 

F 

. Total Neutral 
References^ ' -.03 

Male Refer- 
ences Minus 

Female Ref;. -.16* 

Total Male* ' , 

Act9rs ' -.16* 

Total Female 

Actors .02 

To tat Neutral * 

Actbrs ■ . -.10 

Total MaJ.e ^ 

Roles -.06 

Total' Female • ^ 

Roles ? -.02 'i 

Total Neutral 
Rojes . " ^ r*26f^ 



-U8* 



.,07 
-.02 

-.18* 
-.10 
.09 



t 



.02 
-.23** 
-.04 

.10 



-olO 



.07. 



.14 



[ 



r_.25**' 



-.06 • \-AY.i2^--.-::,^.: 



7-- 



/'^ p < .05 
** = p < .01 



ERIC 



X ■ 




-.07 

-.li 

-.06.. 
^--.04 
-.01 , 
-.09 



.06 

.02> ' 

> • - 



.11 
.08 
.05 

-.12 . 
-.03 

.07 

.00 
-.05 , 

'.go. 



TABLE. 16 

Intercorrelations of Selected Content Factors and ' 
Item Characteristics, Grade 5 

■ - i - 

. ITBS, Language Skills, Level 11 



N - 155 Items 


% 

Male > 
rassxng 


Female 
'Passing 


Sex 
Difference 
in Passing 


Male Item 
iotai Biserial 


Female Item 
Total Biserial 


Total Kale 






— 07 * » 


/ii 

— .VX 


.Ul 


•Total Female- 


nA 




^"".Xj** 


.Ul 


no 
^ 


Total Neutral 
A,cxcrenccs 


i 

• U / # 




— 1 'I 


• 


7.iy** 


Male Refer- ^ 

ciicco rixnus 

Female Ref. 


* t 

.03 


.02 


.06\ , 


-.01 


-.02 


Total Male 
Ad tors 


.06 


.07 ' 


-.05 


-.03 ^ 


-.01 ^ ' 


Total Female 
Actors \ 


■.04 -. 


;06 


-.10 


-.04 

✓ 


.00 * 


Total Neutrstl 
Actors . 1 


.06 

\ 


< 

.04. 


.07 


-.•20** 


-.15* 


Total Male c] 
Roles^ f 


>' 
.05 - 


..03- 


5 

' .06 


-.10 


-.03 


total Female) ^, 
Roles , '/ 


> 

-.11 


-.10 


-.01 


.-•05 


^-.03 


Total Neutral 
Roles I ! 




* .10 


\ -.03" 


. .04 


.04 , 
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« TABLE 17 . . / 

1 

Intercorrelations^of. Sje^ected Content Factors and 
Item Characteristics, Grade 5 

■ • " . ^ / • 

ITBS, Mathematics, Level 11 



N = 71 Items 



% % Sex 

Male Female ' Difference 

Passing Passing in Passing 



Male Item Female Item 

Total Biserial Total Biserial 



Total Male 
References 



Total Female ^ 
References 

Total Neutral j 
References 



Male Refer- / , 

ences Minus ? ' 

'Female R^. j ^ 

.Total feie I '\ 

Actors \ 

Total Female 'i ' 



Ac4t:ors 



1 



.Total NeutraU i j 



' 'Actors 

Total Male 
„ Roles - 



1 1 



Total Female 
Roles 

Total Neutral 
Roles * 



-.13 . 
•21* 

-.24* 
-.12. 

.21* 

.14 
^N/A- 

.19 
-.06 



-.09 
.29** 

\ 

.13 



-.12 

-.37** 

-.22* 



-.27** • .23* . 



-.08 
- .27** 
.18 

-N/A ' 
.26* ' 



-.12 _ 

J 

-.35** 
-.22* 

N/A 



* 



•p < .05 
p < .0*1 




-.07 



-.07 
.05 
.05 

-.08 
-.04 
' .03 

.01<^..> 
N/A 

..14-/ 
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57 



-.04 ' ' 
-.04--, 
-.07 ^ • 

.01 
-•01 
-.08 
-.09 

-N/A -V 
-.04 

f 

> 

-.13 



.It -5 
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( i 



' • TABLE 18 , . 

Intercorrelations of Selected Content Factor^ and 
Item Characteristics, Grade 5 ' ^ 

ITBSi Work Study Skills, Level ll'^'v ''-'^' 



N = 118 Items-'' Male * Female Dif^,ei:enG^' ^ l^e Item Female Item 

- Passing Passing ' lt» Pj^i^ ^tal Biseyial ' T0tal , Werial 



Total Male 
References 

Total Female 
References 

Total Neutral 
References ' 

Male Refer-' 
ences Minus 
Female- Ref . 

Total Male 
fetors 

Total Female 
Actors 

Totai Neutral 
Actors 

'I- 

Totajl Male 
Rol^ 

Total Female 
Rolens 

'Tota). Neutral 
Roleris 



.15 
.17* 
\ -.17* 

£ 



.17* 
-.20* 

-v05- - 
".18* I 
-.11' 



.12 

i 

.14 

.04r 



.02 



.01 



-.48** 



.02 



.00 



-.01 



^.44** 



r.20* 



-.01 



-.47** 



, -.19* 
.-.19* 

V 

.00 

-.18* 
-.18f 

-^18-^ 




iiT ■•■-,i2- 



.13 




-.20* 
-.15 
.07 V 



-,21** 



-.15 



.01 



* =• p < .05 
** 'V P.< .01 



r 



'■5S ' 
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TABLE . 19 



Intercorrelationa of Selected Content Factors and ^V: 

Item Ch'aracteristica, ^ade 2 . ■ . - ^'V7^ 

Metropolitan Achievement Tests; Reading, Primary IP ' i. 



% 



% 



Sex 



N«44 Items - Male Female Difference Male Item . Female Item 
~ Passing Passing in Passing . Total Biserial Total Biserial 



Total Male 
References 

Total Female 
References * 

Total Neutral 
* Re'ferences 

Male Refer- 
ences Minus 
Female Ref . , 

Total Male 
Actors 

Total Feunale 
Actors 

Total Neutral 
Actors 

Total, Male 
Roles 

Total Female 
, Roles 

Jotal Neutral 
%ol)esc^ 



-•05 
• 04 
-.31* 

-:o5 

" .07 ' 

-•93 

-.03 

-.20 

.19 
-.01 



.00' 

:o9 

-.32* 

-.02, 
.07 



.01 



.01 

#°- 

.-18 



.»23 
.02 



-.16 
-.21 
.15 

-.09 
-.03 

-.14 
.03 
-.22 
-.12 



..^2 . 
» 

.08 
-.21 

) 

.17 
-.01 ■ 
.02 . 

-t7 

5 • 

.11. 

i 

-.06 V ■> > 
.22 1 . 
.31* 



.25 

I 

- .13 
-.18* 

-^02 I 
' .05 



.07 



-.06 



.24 



.30* 



* = p < .05 
** f p < .01 



■ o 
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redundancy among content measures, there is^* Increased likelihood of 
relationships for, say "total male actors;" if ^total male references" 
is significant, ^ut even when attention is focused more narrowly oa 

' • V •■ ' / 

the total references only, approximately 20% of the coeffciclents achieve 
signif icance.J ^ 

In view of the frequency of significant , coefficients ,' it se^ms 
there are telationships betweeil content factors and sex 
differences in item success. The general level 'of these coefficients is 
quite low, with most bping in the range of absolute values from .20 to 
w35. . 1?±ve of the 28 significant correlations occur in the 'Grade 10 
reaki^ t1e^t. The others are spread more evenly over the remaining tests, 

i ^ 

On the basis of. these sex-difference correlations, there is some 

support for the hypothesis that total references, regarAess of categor^^ 

(male, female, or neutral) are directly related to relatively greater 

female success. Of thie 28 significant correlations, 23 are negative,, 

? * ^ • • 

indicating a sex difference more favorable tov females for items wftll more 



references. i 



Each of th6 significatit correlations with total female references 



h 

isf a negative value, indicating that the more^ female references there are 

/ I ^ . I .. ^- 

in the item, the greaCer the telative success of female^ oil that item. 
Since female references are mych less frequently encountered than male 
re£erences7\the fact ^hat the|.r presenc^ seems to enhance female perfor- ' 
mance, even tp a very slighfcr extent ,^ is interesting. The results are too 
weak and irregular to posit ^ practical relationship. Such evidence as 

*Signif icance levels 4sed in this study are those deriving /ir6m normal 
—blvariate relatlons.hlps . .A ^lumber of distributions encountered are 
? ;8«fficiently skewed to call this sampling theory into question. 



pport 



■ { 

there is sup^rts the yiew that fanales do somewljat hotter on items with ' 

r ^ •.. . ■ 

' greater numbers of fdmale references; but note that the male, and- neutral 

'"^^l^^^^^ sKoj? such negative correlations in two of four cases, als'o. "* 

Considering male and female rate of suftcess separately, a number of 

'*v^^f ^^^o'^g relationships were found for STEP Reading 2A, Table 6. In 

general', these indicated that itfems with more references were harder 

k^Z^^ 5^1^ female^, and thiWas tr-u^-.regardless of the'Bfpe of 
' V - • * V ' J^. . / / > . 

^^refere^ce; , The Grade 8 ITBS Readrn^JTeet ils^^^sbowed ^ number of sighlf- 

icant, relationships, but with somewhat a reverse tendency: the more the 
references to males or neutrals, the easier for males. 01 the same 
relevant coefficients at Grades 2 and 5,' all are negative. Again, the ' 
patterns are only suggested, but the.wei|ht of this^ evidence is that 
items with references are harder for both males ^an^emales. Females 
^o^relatiyely better^ but both sexes found reference-rich items more., 
difficult. ' • . ' <. 

^ The ^impact on item-total biserial was* studied also. Here the 
results seem so inconsistent from grade to grade and from test to tes't 
that summarization is difficult. -Of the 140 male biserial relationships, 
31 showed a significant level of association between itenf biserial' and 
item content factor.; Of these, 21 were negative, indicating tW more 
references led to 'a 'lower biserial.. , This result- could be artif actual, 

since mor.e difficulty items tenjd inherently to show lower 'biseria^LsT 

. If . ' . - 

(It is trrue that biserial r is theoretically insensitive to item 
difficulty, and Is certainly not; constrained in its range. In the way 
that phi or. point biserial r; ^ would be, v^t- the effects of guessing 
serve to depress the 'mean of the correct response in a manner which 



lowers the coefficient. )j .There is some evidence, as stated' earlier, 
that items vd.th more references are harder. 

While* the same Ipat tern 'is observed for females, the 22 c'oxrela- 
tiQns achieving significance are divided 13 negative, 9- positive. ' 
Clearly, the interpretation that items with more references were less 
highly ' correlated must be b guatded one. / 

' The -computatlSn of item difficulty and of '"item-total biserial wa^s 
based ion groups which exc!qi^ded Che Not Reached subgroup. The presence • 
of spjeed variance complicates sex-difference analyses in a number of ways 
'In most cases the item Analysis shows that very "few items incurred any 
dropout, with only the last third or so at all a^ected, and only the 
last tenth or so very stronjgly affected. A- number of strong relation- 

ships between^Not Reaching and-content were demonstrated, both Cor males 

^ - ^ , . ft * . , « ^ - 

and for females. They tend tp indicate that it^jos with more re^Stences 
are the l^ter items in a tfest, although there are some contrary coeffi- 

- ■ -^-^ . > - !- 

ci^nts. In general, the par^lelism" between males, and females is -such 

that no s'igniftcant performan/ce consequences would seem to ^rise froitf 

' ^- ^ , ' ' « 

... , 

differential rates of work in the tests considered here. Where the 

" " ' .■ ' ' 

significant correlations is aemonstrated for maf^s it is demonstrated 

for females also. It should ^be^tio tied, however , that t'he correlations ^ 

1 ' ' - ^' 

between the Not Reached Inde:^ and the number of references tends to show 

that it'is the later,, more dif f icult ^itei^s that have the most references. 

This finding must be kept in mind In examining the relationships between 

^ ' ^ ^ \ < .1 , - 

content and difficulty which were described .earlier. It is true that.^.. 

^ , ' if J " * • ^ ' ,/ 

items with more references t^nd to be harder, both for males and female^. 
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but this may not mean that they are liarder because they Ijave more refer- 
ences.. As Item writers create more difficult, 'complexly worded items, • 
they may write longer ones, or items which they feel need "humanizing." 
One cannot analyze the data from the present study so as to resolve 
this, question. Indeed, it would be difficult to design an experiment 
which sought to differentiate item difficulty and number of references- 
so that the relationship's could be clarified. 

Sex Status and Perfonnance ^ 

V We -have observed an imbalance of male references to female refer-^ 
ences within items. This imbalance alone constitutes content bias. It 
is, however, what this imbalance may suggest to the test taker tliat leads 
^us to investigate the e^f^ects of content bia^ on test taker performance. 
. ^That is, we are interested* in determining whether the imbalance in the 
frequency of male references t6 female references - is related to sex 

• * • - . 

differences in item passirig. Such a relatioi^iship would iend suf^port to ^ 
the idea that the test t^ker'^ intlerpretation of the' appropriateness of 
the test content for himself or herself — based, in part, u^on*the actuals 

* observed frequencies of ^ales versus* females| in ^the test* items—will 
influence his,*or Jier motivation and hence achievement, 
a But raw or simple frequencies may be mispleading 'when many items , 
contain no references at all; A subsample ojf items containing afly human 
refei^ence was identified* This sample included all items in which there 
were references .to women, men <5r Both, or in which there were male, 

, female. 6r neuter entities, or In which th'ere were any role9. This selec- 
tion created a pool of /i3^s,ecpnd_^rade ltems,,_,2a6_flfth. grade -items, - 320-^ 
eighth grade items and 254 tenth grade items. 

• - ) , s X 

\ 

» • • s ^ DO . 



One critical issue which' we hoped to .examine was the relative status 

•of the men and women i% an item. Since so f6w. items actually were coded 

' ' ^ , . 

as having status dif f ereiices,*- a substitute for this, variable was computed. 



I tems^n which male entities outnum^re^ female entities male entities 
equalled female entities, and female entities ,outnumber^|^aale ei\ti£ies 



were identified. Table 20 reports the.Rejrcent of thejjptels at each grade 



level havingfthese (jKarSc^eri^l^s'. ^At all grade ?eifl^i^,,^^^^ms in which 
female entities outnumber male entities are the f eweSt, *^nging from 
appfoximatel)r-2"l% of the items at Grade 2 to approximately 5% of the 

itetns at Grade 10. Items, in which male, eniiities outnumber female entities 

^ ' , • • s * ; 

on the other hand, constitute the bull of the items, ranging from appxox- 

' 1 'i^ " ** } 

imately 54% of the items at Graded 2 to approximately 42% of the items at 

Gra^e 10. . Only in this latter age rang« do the number fcf items in which 



males and females occur in equal numbers^ outnumber those in which males 
outnumber females (53.2% to 41. 7%) At 'all other grade levels, items * . 
with more males' outnumber those^^which treat the sexes equally. ^ 

In order to investigate the effect'of this sex imbalance of content 
on test ^aker performance^s^^ a multiple regression^analysis was undertaken.^ 
Other variables thought to have an effect on test taker^perf ormance were; 
Included along with tWQ, dummy Variables representing the relative status 
of males and^ females in the it^-. Table 20 lists the variables 'and ^- 
their ^measures^^ The yariables fall into' ^^^^ large categories:.^ achieye--^ 
ment content area. dummy variables, a dummy variable ^f or a second source > 
of test Items, it^m characteristic variabies, and ^jp' status^ variables. 

From Table ^21 it is evident that achietvement' content area and item . 
cliaracteristic variables are highly related^ to sex differences in item , 
performance. The relevant comparison in the ^content area is the "language 
arts" content dummy, • * 64 



Table 20 



Percentage of Items at Each Grade Level 
in Three Status .Categories 



More Female than Male 
Entities (dummy) 

F^ale Entities ? Male 
Entities (dummy) 

- 

More Male than Female, 
^ Entities (dummy) 

Total- 



Grade 2 i;gade 5 Grade 8 

K 



20.9 

25.6 

53.5 
100 
. 43 



15.7 

42.0 

42.3 
100 
286 



16.7 
;35.9 



'47.4 
100 
320" 



Table 21 



Standardized Partial Correlation Coefficients 
for Variables in Equation for Each Grade 

Percent of Males Passing Item Minus** 
Percent of Females Passing Item 



Item Conteixt Predic tors 
Reading (dummy) 
Math '(dummy) 
Social Studies (dpyjmy) - 
Science >( dummy) ^> 
Word Study Skills (duirany) 



Grade 2 



Grade 5 
|,13*> 



.22*** 



Grade 8 
,58*** 
.32*** 



,32*** 



Grade 10 

.33*** 
. 44*** 
.57*** 
~737*** 



Test Identifier / 

Calif orilia Achievement 
Test (d\iirany) 



Item Characteristics 



Other than Xext OnL] 
Length of Item 



Status Indicator 

More Femi^le *Entiti^s 

than Male Entfitie'B (dummy)^ 




Female Entities = Male 
Eptities (dummy) 



■4' 



N= 



-12 
.27* 

I ■ 

.13 

t — 1 _ 



-.14** 



-.09* 



\20 
286 



. 16*** 



-.10**. 



V-03 



.31 



320 



,07 



-.07 
-.07 



-.07* 
■".06 

.41 

254 . 



*p < ..10 
**p < .05 

< .01 



66 



f -58- 



At all grade leVels, however, significant sex effects are found. 
Because the direction of the effect Is different between the second 
grade and all other gradea, some caution In Interpreting the^ ef f^ect miisV 
taken. Since the number of Items at the 'second grade level^ls'^^so 
|\^mall, this effect may be? an artifact. • . 

At the flYth, elgjith and* terith grade* levels the effect Is In the 
direction predicted. .It must;i'be held In mind that.ftems In whlth more 
r^les pass the Item tl]an do m^les will have a negative number for the^ 
dependent variable. Thus, "a higher 'percentage of females than males 
pass Items ^contalnli^ more female t^ian male entitles or equal numbers 
of male and Ifemale entities. This Is the case after the effects of 
Item achievement content, format and Item^ length have been partlalled 
out. . ■ ^ ^ * ■ . i • 

» Noncontent Factors 

^e primary fociis In the study yas concerned Vltlythe Impact of 

/^content on the statistical properties of test Items. These statistical 

^ propertllgsV, however, are Influenced by the characteristics of the group 

taking the tfest, and dt would be possible to err In attributing sex- 

^ linked performance differentials to' content alone/if the items were 

organized ah4 presented in /certain ways.' 'That is, c6nter\t-sex relatlonr 

ships might be an artifact ^of ""tf more^ct^mplex model in which*cont^nt is 

correlated .with, some characteristic of test'organization and'^thls cji^arac- 

teristic of^ the test is in turn^ related to differential performance by 

the sexes. , / o /^"^ ^ 

' ' / ^ . ' „ ' 

^ ^ A good example of thYs' possibility' ndght be a "situation in which tlie 

V ' ' / ' ^ 

items with the ^greatest amoun,t of ^sex-linked content are placed in the 



later^'Stages of test as the last items. These items are the ones^hich 
are mopt influenced by ^variation among respondents in the rate of work. 
If the sexes differ in rate of work, fewer individuals of one sex will 
be working 6n these lare» items. The shift in number reaching may 
inf^ence the statistical descrijAiion of the items, and the stage woul^'" 
be set for a misattribution of statistical differences to content factors 

A similar difficulty is presented if there are sex differences ill . 
the tendency to omit. ^It is reasonable to asfeume that .at any given 
level of ability the tendency to omit is^'a function of the difficulty 
of items: that -harder items are mote likely to be omitted than easier 
ones. But if the sexes differ markedly in a tendency to omit, theyi^ 

^the sex with fewer overall .responses will probably show less "success. 

. The other sex will show a greater percentage passing, for, while ^ 
relatively large numbers ,of thew dd not know the 'answer, the incidence 



of correct responses due to chance alon^'^may be appreciable. Thus, 
the more difficult the itjem, the greater trhe possifcility of a sex 



difference due to , differences 1^ 'omitting. If the 'or^nization of the 
test is such that content-rich items tend, in fact to be the more 



difficult inems , the stage.*again is set; for the misattribiltion of 

^ '» ' ^ 

_ V % - y . " 

statistical differences to content differences. ' 

s^Some aspects of .these technical ems were reviewed in the 

'odiscussion of the' correlational analysis.'* It was 'f eft, however, that 
potential sex differences in the tendency to omit or in the character-' 
Istic rate of work were ^ themselve^s*^ariables of some interest. 
number of psychdlogical characteristics are attributed to the se^efe in 

cominon stereotyping. Risk- taking, for example^ is a characteristic more 

' ^ '''' . 

often attributed to males than to females. Sincfe risk-taking might " 



*logicall]f relate to the tendency ^to omit, or to the tendency to m^k 
the last items in a test in a flurry of speed, an explicit descjiptioix 
of both tjie Omit measure and the Not Reached measure was developed.. 

Table 22 sh9ws all tests which had any item" with' a percent omitting 
for either sex -greatefi than 1%. Any test not specifically cited in* 
this table had^no items"" MtR ^ percent omitting , as great as this. - For 
the tests which had ^ome of these items, their* frequency for niales a'nd 



females is given. Further, the subset of items which showed a tendency 

to omit greater tlian i%» is "presented. ^ . » ' 

^ i , / - 

Thfe general conclusion from ^his table and from the tests that had 

absolutely no such items is that the impact bf omissions is trivial at 

any level, and that there is no demonstration of "^^^ sex 'difference in 

this ten)jency. It is true that the-' math tests show more .frequent '^female 

omitting, but very few of tl)ese items reached Ibhe 2% j/vel. There^ is 

* 

certainly np^upport for a risk-taking differential between the .sexes. 
Tables 23, 24, and 25.provideti descriptions of the Not Reached 

'''''' ^ w ^ ^ 

variable for males and females at^lrwo points in each test: th'e last item 
and tU\e fifth ftom last item. While ncL standard f^jdex or convention has 
been -established within the field of measurement .by which to evaluate 



such data, tiie levels are such that most test workers would probably 
j'^^e them to be unspeeded. The comparison of the 'sexes 'indicates that 

.level of difference -between males and. females is so smair that no 
major imp^t on the itetias could be attributed to it.. There is sbme . 




evidence of correlations between content area and rate of work (more 

* / . -r- — ^ . ' ' 

females reach la£er verbal items, for example, more males i^e^h laj:er 
mathematical items) but because' the differences themselves 'are small,* 
the patterns have little intrinsic interest; ' v 



Table 22_ 

Tests with "Omit** Response 
% 

• Total 

' Test . . . .Items 



California Achievement Test, • 



Math Problems 




15 


JTE? Reading Part II 




30 


STEP 'Mechanics of Writing <^Part IL 


45 


STEP Math. Basic Concepts 




50 


STEP Social Studies Part I 


s 


35 


STEP Social St;pdies Part II 
« » 




35 . 


STEP Science Part I 




45 


STEf Science Part II 


4 


45 


ITBS - Level 14 Reference. Mat er.ial 


59 


ITBS - Level 14 Math Concepts 




48 


ITBS - Level 14 Math Problems 






ilBS - Lev^i 11 ^)^cab:i4ary / 






ITBS - Level 11" Maps 


« 


36' 


ITBS - Level H Math Concepts 




42 


ITBS - Level 11 Math Problems 




.29 


MAT - Sentences. - " 
}ikT - Stories . . 




13' 

31 ^ 



Greater than 1% 



Number of 


Items with 


Omit 


> 1% 


Omit 


> 2% 


Male3 


Females 


Males 


Females 


- 

3 


• 7 


0 


- 

j2 

i 


1 

X 


Q 

y 




i 


0 


V 


^ 0 


Q 


11 


' 20 


0 


9 


- 2 


2 


0 


0 


10 ^ 


6, 


• 6 


. 0 ■ 


% 6 


7 


1 


3 


0 


1 


ft 

.0 


' \ 


2 




, 0 


''■0 


/ 




« 


^ 0 


•5 


io , 


0 


1^ 


1 


2 


0 


* o- 


0 " 

* 


3 


0 


0 . 


3 


6 




. 0 


l' 


2 


p 


0 


■ 2 


0' 


0 ' 


.0 


24. 


15 ■ 


6 


2 



Table 23 ^ 



'Not Reached" Comparisons 
Males Versus Females 

Grade • 5 



Content 



'% Not Reaching 
Last Item 



% Not Reach ijig^s 



5.th from Lasfc>Jt;e^^ 



Iowa Tests of E^s±c Skills > 


. Males 


Females 


Males 


Vocabulary 


8.6 


* 

^' ■ 5.8 - ^ 


5.2 


Reading 


1Q.4 


* 7,5' " 


8.8 


Spelling 


13.7 


' "8.1 


8.6 


Capitalization 


8.4 


4.3- 


• ~ ^ 

4;7 


Punctu«:ion 

% 

Usage ' 


2.6 
2.3 


1-1 

^ v*. ^ ■ 

^^..0.0 


' ' 1.4. 
1.1^. 


Maps, 


19.2 


23.9 


10.4 


Graphs ' „ 


6;6 


5.8 ■ 


2.6 


Reference Mater.iala 


, 17.6"- 


13.8 


14.9 . 


Math Concepts 


■6.9-' 


-•,..'4,v "7.7 = 


2.5 - 


Math Problems > 


11.6 i 




3.3 


*• * 









Fema les , 
2.7, 

i ''v^5,.4 



4.3. 



2.1 

■ 0.6 

0.0 
12.0 
2.3 

r. 

3.7 

3.6 A 



1 



Table 24 



Cdnterit 



'/Not Reached" Comparisons 
Males Ve'f^us Females 



4 



Iowa Tests of Basic Skills 

Vocabulary 

Readi^ag 

Spelling 

Capitalization 

Punctuation 

» 

Usages 

Maps " * • , 

Graphs 

Reference Materials 
Math Concepts : 
Math Problems ' > 



Grade 8 



% Not Readhing 
Last-^em 



% Hot Reaching 
5th from |.ast Item 



Males 


Females 


Males 


2.4 • , 


_^ -1.6 


1.6 


6.3 


5.7 


5.1 








8.5 


4.9 


5.5 




0.0 < 


2-. 3 


1.6 


0.0 


J. 6 


0 .-a ' ■ 


0.0= 


0.0 


11.8 
4.4 •■ 


c 

14.2;S 

3.4*- 


. 7.1 


7.3 ' 
6.9 


*5.5''' ' 


' . 3.5 


13.9 


\, . ' ' 
, 18.9 


' 6.7 



Females ' 

ito 

/.4 

3.2 

0.0 

0.0 

Q.O 

6.5/ 
' 1.5 

2.^ 

1.5 ■ 

7.7 



Table 25 



"Not Reached" Cpmparisons 
^Males Versus Females 

Grade 10^ 



Content 



Sequential Tests of 



Not^ Reaching 
•Last Item 



% Not Reaching^ 
5th from /Last Item 



* Educational Pro^^ress 


Males 


Females 


Males* 


Females 


Reading Part 1^ 


2.3 


2.4 

1 




1.0 * 


Reading Part II • 


20.1 


25i4 " ', 


•16.2.^- j 


.19.7 - 


Mechanics of Writing Part I 


1.4' 


0.0 


0.0 


0.0' 


Mechanics of Writing. Part II 


48.8 


50.2 


. 48.4 ■ 


49.9 


English Expression 'Part I 


2.5 ■ 


1.6 • 


1.3 


0.0 


English Expression Part 11 


■^33.6 


32.1 


16.8 


^ W.5 


Math Basii^oncepts 


12.5 


19.0 


6.2 


, 8.5 . - 


Social Studies Part I 


3.6 • 


4.2 . 


.1.3- ' 


1.8 


Social Studie-s Part II 


18.7 • 


' 2:6 .0 




7.1^ 


" • / 




• ? 






■California Achievement , Test 











Mathematics Problems 



.12.2 



1.2- 



16.3 



1.2 



— * 

I 



Implicit Analysis ' 

The project centered bn the explicit analysis, of the tests, {by 
^hich was meant that the measure "of item content would be the frequency 
counts of explicit references* Bflt it recognized the nAed for a consid- 
erate^ of , what was called^mplicit analysis. Such an analysj-S could 
not be\arried out* f^ all of 'the thousandp of^'items contained in the 
tests undefS;«yiew. Sor, indeed, would it be a meaningful prd^ess for 



- 6 



judges in many ^ases. Much of the material was sex-linked only at very 
abstract levels of cultural knowl^ge. ^ The ITBS material on Study Skills, 
for example, relating t^ Graphs^ might be generall]^ seen as more related 
to- male interests an^ experience by application of a stereotypical logic 
concerning males and Quantitative pursuits. But this would .not be an 



implicit analysis o^ conJ|)|riW in' the sense suggested here. Accordingly,* 
the implicit analysis was.^jb^^icted to the Grade 10 sample and then to 
the reading malferial ^only^!^ ^e ir^ading material, richer iiT referential * 
content, seemed a more'^pprop»r;Late focus for thtf activity. 

- •. / w ' ' ■ ' 

' The concept of *att^imjJl±cit; analysis derives from the hypothesis . 
that the content* of tests may be ^ex-linked in ways other than through 
explicit references, such as through culturally determined opportunities 
to leajm.' Females t:erld<i^tQ^o Uetfter on aesthetic ox artistic content, 
males orf scientific or oA jaechanical knowledge* Thus, the propoifClon — 
of tests -which Is dieted to these categories may^/e an Important deter- 

nei- o£ sex differentials. / The studies by iCoffman (1961), Donloti (1973), 

, ^f-i ' * . • 

arii^\S^aasberg-Rosenberg and Donlon (1975) all demonstrate the potential 



u%llfy'^fc;uch impli 



cit factdts* 



Ei^^^i^ers^^t^^ identified to capcy out the implicit analysis. 



\ 



:ERIC 



All were cq^^^^graduat^^^ There were -five females and three males. 



Recognizing the futAltH^f generalizing ^rom the sample of judges, a 
: great effort was not made to reflect^ the characteristics, of any popula- 
tion. The activity of ^he analysis was seen as a demonstration activity 
which tested the utility of the Implicit- approach less formally than 
* might a statistical model.* '"^ . ' . - ' " ' 

The choice of highly educated judges .--aftd impiicltly, highly 
Intelligent ones, seems senslblj. What is being sought in this kind 
, of aiialysis Is the application of knowledge of a" culture. This knowledg 
is heldjmost^ strongly by thos.e who by virtue of training and intellect 
have considered the impact of sexuality at length. But.^t is' recognized 
that there, are no perfect mirrors to culture. One must be in a culture- 
to know many -6f its dimensions. Being in it, we surrender an , essential 
• .qljrjectlvlty. . , 

Th6 .devising of appropriate. Instructions for the implicit analysis 
was difficult. Wle the judges were in general soBbisticated about 'l 
. tests, the judgments which they were asked to make would draw upon the/. 
l more tenuous characteristles of the language in tests. The instructions 
had tQ provide sufficient structure so that the judges' internal scale^ 
. of the sex-referential "dimensions of language were approxlm9t,ely in line 

with each other,, but avoid prejudicing the group in the direction of 
V; perceiving a content imb^iarice of any kfnd'. The^f ollowink^^wjO^'lnstruc 



tions were prepared: 



In connection with^afTNIE pfojecjf: on sex differences , we ' 
, ar^ conducting, a comparison of the Wrformance of males aqd 
--rfemales on a variety of achievement' tests. Currently we are in 
^the process of relating performance data to content character- 
istics. This involves an analjr'sis of both- explic~it and implicit ' 
item contents ^ ' . . 
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■ . '» . • 

Content charapteristlcs are explicitly determined by 
counting references to people: males, females, neutral. Thus, 
J the item ) * > • ■ ' .• . » . 

The boy and his father went to the zoo. 

* would be jlidged to have more explicit male content since it had 
two references to males and no references .to female or neutral 

• .characters. ' , , . ^ „ 



A different' approachM:o content analysis relies more oti the 
judgment of- implicit sex cji^f f erences. Thusr, a* vocabulary .item 

The casserole was cooked too long. ^ 

has no explicit sex- referential content but- might be judged* to 
be more readily learned within. female cultural experience. 
Judgments on implicit content should fall into one of the 
following categories: . • - 



1. masculine— content mor^ readily learned within 
the male cultural experience . 

2. feminine - content more readily learned within 

/ the female cultural experience 

\ 3. neiftrai - conteivt relat^^d to the same types of 
cultural experiences for both sexes 

4/ balanced - content favoring one sex ar-e* offset 
]^ other characteristics favoring 
. /the other sex. , '. 

' ? "-^ . . ' ' " ' 

In "some items, however, the analysis o^ 'implicit content 
* will/ not be so clear-cut as in tKe preceding example.' Sometimes 
the content of , the item may" be set in a masculine f ramework, 
while the skill required for successful completion of the item 
•is more readily learned within the female cultural experience. 
Consider the following vociabulary item* 

No mathematical theory was too abstruse for this 
_ man who had spent mdny years studying the subject. 

Although the item deals wi(.tb more masculine interests, the vocab-' 
ulary word asked may be more familiar to women. In cases such as 
this,.' it is .up to you to determine the predominant emphasis of ' 
. , the item. . / ^ 

Sensitivities^ to^ sex dlffefences in cultural experiences vsiry**^^ - 
with the- age of the child,^ hence it is important .toJkeep this-' ^ -fp: 
dimension In mii3^d when rating items. For^fhe purp^o^es of this "^^'''C 1 
analysis.^ you are being asked to cons^er-. this content with sttidents^ 



in Grade, Ten in mind. ' f - . -X?"^' -Si^- a j-/^ ,4/ 



We would appreciate^pun help in rating the follo^Sng " 
^ items and reading passages.. There are^no right or wrona 
" answers; it is merely .a matteiJl^of personal ^judgment._ As you 

p . 4eveig||>. questions and/or comments, we w5uld appreciate them 
in wfiting^to kid in* the development of a^brijff "implicit 
' "analysis" manual* * ^ - 

Thank yo^ for youi|^t!o6peration. 

' . These instructions , were' responded to by some judges with appeals 

for greater definition. ' These appeals were' received sympathetically 

with expressions of understanding for the difficulty of ^he task and 

the general problem of "What do you mean by the male cultural experience?" 

^but ythe judges were- encouraged to attempit the task aimed only with 

this level of definition. * . 

. ^ The results of the analyses are presented in Table 26 which lists 

0 the judges' categorizations as Male, Female, Neutral, pr Both. "Neutral." 

of course, is- distinguished from "Both" in that .it has, in thfe opinion 

>f the jjudges, no 'serospecific experiential background. "Both" implies 

. a kindfqf separate but' equal androgynous background » J 



|F The^ data may be evaluated from several Viewpoints: Their internal 
* ' * * 

'consistency, ^"heir relationship to the er^plicit'^nalysis , their relationr 
ship to' the statistical properties of the items, "and, finally, a compair- 
ison of the relationship of implicit and explicit analyses to ^the 
statistical outcome. ' ^ . ' * ^ , 

With respect to internal consistency it is evident that there is a 
generally high level of^a^reement among the judges. Somq of this derives 
from their ^common tendfeJjAy' to designate Neutral responses. Such .responses 
_are_designated in 2$9 pf A§,0 instances. Particular attention must 
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Table 26 ' '» V 

Edges' Classification of STEP Reading Items for IInp^Licit Sex Content 



ERIC 



Part I\ 
Male Female- Neutral 



-Both 



Part. II 
Male - female Neutral ' Both 



1. 

2 


8 

4 ' 




4 


— 




I 

•2 






5 


3 
4 


5 

' ' 2 




3 

6. 


— 




' 3 , 

. 4 


1 ' , 


1 


7 

- 6 


5 
6 


~ 

7 


, 8 


1 


- — 




5 , 
6 


1 v ' 




6 
7 


7 


4 


1 


. 1 


2 




7 


t 


2 


4 


8 


3 




. • 5 






8 






8 


9 


2 




6 
7 


1 




. 9. 

10 


2 

> 

1 


1 


6 
•6 




3 


.1 


2 


2 ' 




^\l ■ 


-4 


— 


8 


12 • 






8 ^ 






12 


2 " 




6 


13 




6 


, 2 






• 13 


2- 


" - 


6 


14 


: - 4 




4 






14 . 


2 


• 1 


5 


15 


/I 


/ 


7 






15 


3 




5 


16 


4 


" 1 .. 

'l 




- 




^ 16 


4 




4 


17 


1 




^- 1 




17 . 


6 






18 






1 






■■ 18 . 


. 5 , 

t 




3' 


19 ' 


3 




, ' 4 


"1 ' 




19 


(ft 

3^ 




5 


20 


4 




3 


1 




20 




1 


7 


21 


1 


— 


7 






21 


3 


r 


5 


22 


6 




1 


. 1 




• 22 


2 




6 


23 


• 6 




^ 2 




23 


1 




7 ' 


24 


1 

• 


7 








.24. - 


2 . " 




'6 


25* 

26 ' 




4- 

1 • 


4 

^ 7 
3^ 






25 

,26 




3 


3 

,8 


20 


4 




1 




> 27 




» 1 


5 


29 
•,.30 


1 

2 

, 1 


6 

. ■/ 

2 


- 4 
■2 


2 
1 




28 
■ ' 29 
•^0 y 




_4 
3 

'3,1 


4 
5 
5 






■ r * 


















1 
1 
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focus on, cases of judgments which are diametrically opposite: one judge v 
. says- the content implicitly fa vors malps, annfbpr sees i t asi favoring 

' females. For 13 items this was the case, but in ^all but ^two cases nb 

'» . ■ , . * ' 

pore than^ne judge reached one of the two ppsitiorrs. Items 2^ and 30 ' * 
in Part I, however, drew M-F splits of 2-6 and 3-2 Respectively., Each 
of these conflicts . seemed to grow out of the juxtaposition of specific - 
adjectives which were <:onsidered more , frequently known by females, and 
jiouns, which/in the cont.exts o^ the stimulus sentences^ were considered 
more JrequeQjtly known by males. ' • - ^ - 

The line of consisten^cy between implicit analysis and explicit 
analysis ^is more difficult to^ sy^mmarize statistically. Each item is 
described by a vector of four numbers in the implicit^.analysis and by ' 
three in| the "references—summation of the explicit analysi-fr;*^ To simplify, 
attention is fpcused on extreme cases. Eight Items were judged masculine 

in content in the implicit analysis by'fdfrye or ^ipre concurring 'judges. 

• ^ ^ — - — 

Six of those eiglj^t items were vocabulary items from among the thirty * 
items in Part I. While four of. these items were in the ^top third of" all 
items in terms of mal^ references, four had no male references whatsoever. 
Four itema^were judged female by fiye^r more concurring judges ifi the " 
^impllcit analysis. All were in Part I, the vocab^ary s^cftioh. Two of 
these had the stirongest explicit content values; two hltd no reference at V 
^.al^. For these twelve items, accordingly, s^ ha^greater than average 
explicit values in the appropriate direction while six had no direct ' ^ 
itial content at all. The judgments in the implicit analysis. .were 
£lX based on characteristics of the' items not reflec^ted inT the 
exptLclj: analysis. * . * 

The evaluation of the relationship between the i^^^^analysis. and . 

. • * . ^ * ^^^^ 

the difference ia item 5if f icult^between the* sexei^^|^ade by converting. 
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judges' ratings 'into predictions. In order to. do this 22 items were o 

* 

identified which had three or more judges indicating a judgment in favor 
of one sex. Item 30, Part I was not included in this analysis becuase 
of the evidence of conflict amon^ the judges. 0^ thesfe 22 items, 17 
predicted male superiority, 5 predicte^ female superiority. Th6 ij:eni 
analysis indicated that the 60 items were actually divided as 28 easier 
for males, 32 easier for females-. Table 27 shows the resultijrg contin^ 



gency t^ble. 



Table 27 

C 

Joint distribution of* Item Assignment 
by 'Judges and by Item Analysis 

Judges' Assignment 



Item 
Analysis 





F > H 


M > F • 


M = F 




F > M . 


* 

4 


7^ 


23r 


32 , 


M^> F 


' 1 


10 


17 


23 


5 , 


17 


"'38 

<* 


■ 60, 



X = 2.6*4, p > .10, 



/a 



This contingency table indicates a weal^a^siyciation, not statis- 

^tically significant, in support of judges* , ability to predict the direction 

* " i 
of item difficulty. This level of prejjiictiorif iis to be contrasted with 

the levelg. reported for . the correlations oi| thj^xpjiclt analysis. For 



the STEP Reading Tests these? were ^g0n6rali3f/d#^ the v low .20s -and modestly 



significant. It indicates that tHe iifc|^i^g^iFjud^ents were 'probably hot 
as f)redictive as the explicit on^. j&idges were able -tc 



:o make decisions 



about the material^ and tq^demoristtate a fair level of agreement among 
themselves, but/there is limited evi(jeri^!e^tjiat the items they Identify 



differ stat>iscicaMy- frpirf t^o§e they do not 

oU 



r 
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Table 28 



Error Analysis' for MAT Readings Primary II 
Percentage' of Observed" V^ng. Answers for Each Possible Wrong Answer 



PART I 



PART II 



Sentences 



Answer Choices 



Item 


Sex 




2 


3 


i . 


..M • 




. 47 


IS 

■53 




F 




' 41 


59 


2 • 


M 


■•58 


42 






F 


57 


43 




3 


• • M 




"63- 


38 




F • 


~-'m 


70 


30 


4 • 


M 




48 


52 








,38 


62 



M- 
F 

M 
F 



54 
54 



« 

46 



63 
64 



37 
36 



r 


M 

' F 


53 
52 


47 
. 48 


_ \ 




8 


M 

' F 


46 
' 49 


■ 54 
51 






9 


. ' M 

F 


_-44 




56 
58 




10 


M 

• F 


56 
64 


44 
* 36 








M . 

* " F ' 


67 
70 




33 
30^ 




12 


■ M 
F 


. 44 


36 

- 57 • 






13 


M 


40 
33 


60 
^' 67 







I 

Item 



10 



11 



12 



13 



Stories 







Choices 


Sex 


1 


2 


_3_. 


M 




,73 ' 


27" 


F , 

1 




69:^- 


31 


M 


77 




23 


F 


' 81 . 




■19 


M 






F ^ 


49 




51 


M ' 


47 




J J 


F 


'45 








- 

32 


68 




£ 




71 . 




M 
F 

1. 




52 • 

52 f 


48 
48 


M 


-44 


56 V 




F 


43 


57"^ 




M . 


44 


56 




F 


42 


58 




M » 




53 


47 


F 




51 , 


49 


M ' 


45 , 




55 


*F 


.. ,38 




62 






46 


54 


F 




4S 


57 




53 




47 


F - - 


-46—. 




-54 






57 


43 


F 




56 


44 
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^ , ■ ^ -Table 28 (cont'd)^ ' ^ ' ' 1/ 

Error Analysis^ for MAT Reading Primary II 
Percentage of Observed Wrong Answers for Each Po^sible^rong Answer 

PART II ^ PART II 

Stories * Stories ^ 

Answer' Choices Answer Choice^ 



14 


M 

' r 


1 

X 

67 

/ J 


9 

33 

9 7 




15 


M 
F 

* 


. 64 
75 


' 36 
25 


> * 


16 




« 

50 




50. 


17 


M 

V 


48 

JO 


52 




18 

• 


M 

* T? 
r 




■ 72 

A 7 


28 


19 


M 

T ' 


•49 

42 


— 


51 ' ^ 
• ■ 58 


20 


. M 
F 


66 
71.. 




«4 

29 


21 


M 
F 


41 
44 


59 
.56 




ir 


. M • 
F 


69 




31 
27 


23 t 


M 
F 


66 
73 


• 34 
■27 




24 


M 

F . 




49 
' ■44s 


*« 51 

56-' ■ 


'25 - - 




58 




•42 '\ 




F 


57 


♦ ^ 




•26 


M . • 

F 


> 


. 43 
36 


57 
64 


27 


M • 

¥ ■ 




59 
60 


41 
40 



f . , » 

Item Sex ' 1 . 2 3 



61-39 
" ' F - 68 , 32 



X' 



9 

* 29 M 68 - . - ;3^2 

F 69 - 31 

•30 M 63 37 

F : - . ' 63 37 

31 • • 58 ■ - , ' ■ 42 

F -64 - 36 



8 



Table 29 

i 

Percentage of Observed Wrong Answers for Each Possible J^rong Answer* 
PART I ^ . ^ ^ PART II 



j - Error Analysis^ for STEP Reading 2A 



Answer Choice . Answer Choice 



Item 


Sex 


1 


2 


. _3_ 


4/ 


Item 


Sex 




2 


3 


: 4 


1 


m' 


66 




26 


9 


1 ' 


M 










F 


67 


— 


27 


6 . 






, 12 


34 


55 




2 


M 




2°8 


•29 


42 


« * 

2 


M 


2 




J J 


91 




E 


— 


23 


32 


45. 




F ^ 


"26 





,,'53. 


21 


3 . 


M' . 


17 , 


41 


43 






M 

n 




9 7 




1 A 




F 


29- 


30 • 


42 


— 




F 


— 


.30 


50 


20. 


4 ' 


M 


30 


42 








M 

n 




DJi 


9 A 


9< 




F 




4-9 


«... 






F 




48 


23 


29 


5 


M 


27 




28 




S* ' 


M 
n 


Z^ 


A 9 


7 A 






F ^ 


13 


— . 


32 


55 




F 


28 


"44 


° 28 




• 6 " 


M 










0 


\f 
fi 




9n 








^ F 


— 


■43 


32 


23^ 




F 


29 


18 


53 ' 




7 


' M 


25* ' 


23 




S2 


7 


M 








9 7 
Z J 




F . 


29 


24 


— 


46 


F 


33 


41 




26 


8 ' 


M " 




29 


41 


30 ~ . 


8 
o 


M 

n 




1 8 




AO 




F 





15 


61 


: 22 




F 


— 


19^ 


34 


■ 47 


t 

,9 


M 




40 






Q 








A A 


97 
Z / 












Jo 




r 








oi 


10 
























M 




52 


36 


13 




M 
ri 












F . 




57 


37 


6 




F ' 




57 


32 


\ 


11 


M * 


23, 


11.* 


66 


t __ 


11- 


H 


J 35 


23 


.42 


I: 




F 




5 


72 






•5 • 


39 


►20 


41 


'_V 






13. 


.74. 






12 


M- 


^ ;27 


32 








18' 


.80 








■ -F .' 


*26' 


37 






13 


. • m: • 






' 50 


23 ^ » 


13 


*r ■ 






39 


23 








1 


52 


29 


»• r • ' 


F. 




35 


39 


26 


14 


. M 


21 - 


60 




■19 ' ■ 


14 


.M 


i6' 


50 


* 


35'. 




;f 


17 


•58 


ku--- 


26 . • . 




F* 


.15 


45 




40 




* 


















15 


M 


• 28- . 




.5,4 




.15 


M 


• 30 


48 




- 2,3 






34' ■ 




54 • 


••^2 . 




^F 


34 


•44- 




• 22 . 



J 

ft 



.7-5- ^, . --- 

: Table 29 (cont'd) " . . 

. ' ■ Error Analysis for STEP ' Reading ^2A " . ' ' . 

Percfentage of Observed Wrong Answers for Each Possible Wrong Answer*' 

■ k i - ■ , . , . • 

■ ■ . » . ■ • - • : 

. PART' 1 PART II / 

T* ■ — ' 1 

Answer^ Choice' Answer Choice 



Item'" 'S«c 1 • 2 3' - 4, .Item Sex ^ 1 2 3 ' 4 



16. M " 32 . 28 40 ^- 16 M 19. 31- 49 

« F '24 25 51 — • . F 21 29 49 

17 M. k — 26 34 ' 40 ^' _ 17 M — 34 49 .17 

F 19 28 52 . ^ F * — 39 44 '^17 



18 M y. 33 -- 23 43 '18 M 30 — 42''^ ,28 
F . 24 -^^=-7- 17 59 - F. - 25 , — 45 30 

,! ^ ""• • 

19 M " .425 52 ' 23 — 19' M 32 ,35 33 — 

V F 31 38 '31 — 

20 M 2— 31 . 47 » 22 _ 20 M 31 1- 38 ' 31 

. , F - 33 ■ — 31 , 36 

'^-^ 21 . M 5:?- — ' 23 ^19 21 M 18 ' 27 55 

'• F id 21 63 — 

22 M 22 -•63 ,15 ^2 ^M. 21 — 30 . 50 

F 21 • -T 26 *i53 



23 " M . 31 15 53 — / ' 23 M 30 34 — 36 

F 2:3 34 — ^3 

24 -M* 66 — 26 8 ^ 24 M — 29 37 34 

■ F — 26 / 36 38. 



M 


J 25 


52 


23 




F 


-.•26 




21 


— 


M 




31' 


. 47 


» 22 


F 


32 


40 


, 29 






















M 




— 


^ 23 


19 


F 


» 53 

V 


— 


21 


25 


M 








• ' 


F 




28 


57 














M 


- , 31 


15 


53 




F 


29 


13 


■ 58 




M" 


66 




26 


8 ^ 


F 


72 • 




22 


6 


M 




40 


36 


24 


F 




37 


38 


• 25 


M 


29 


42 


.29 


< 


F 


34 


.^1 


25 




M 


25 




• 54 


'21 • 


F 


23 




57 


20 


fi 


^ .28 , 


40 


32 




F 


29 


'44 


28 




M 


30 


48 


22 




F 


•■■ 31 


49 


19 




M 


■ . 11 


t 


80 


9 


F 


16 




75 


9 



25 , , M — 40 36 24 W • 25 M 34 ' 36 30 

\ F 

26 M 29 42 29 — 26 M 35 34 - « — 30* 

^ . .F 37 27 -.— • 36 

27 ■ M 25 — ' 54 '21 • 27 ■"■ M , — » 36 40 24 

, PJ^ — 35 40 25 

28 . 28 40 32 — . '28 M • »24 50 — * 26" 

.• F '23 49 —. 28* 

29 M 30 48 22 r- 29l' , M 35 . ' 39 '—'=..26 

F - 36 37 — ' 27 

/' 

30 M . 11 •'— 80 9 ' 30 * M 24 41 35 

■ F • 22 ' 46 33 



1 


2 


19" i 


31 


21 


29 




34 


— 


39 


30 




25 , 


— 


32 


35 


31 


38 


- 31 




- 33 ^ 


— 


18 ' 


27 




21 


21 




21 • 




30 


34 




34 




29 


— ■ 


26 


34 ' 


36 


40 


28 


35 


34 


37 


27 




36 




35 


»24 


50 


■ 23 


49 


35." 


39 


- 36 


.37 


•24 


41 


22 ' 


46 



Summedjpercentages may differ fi;pm 100 due to rounding, 
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. ^ Zn general, the judges were not comfortable with'^the processes. 
They sensed many of the ambiguities they faced-. In fact, totie judge . 
. explicitly reported an awareness of the 'conflict /betVefen feminine adjec 
tives and masculinfe content, -An example^of this /conflict might ^ie \ 
"Their knowledge 6f alge^braic transforma'tions was precocious." It?ems" 
and t^st. cont-ent in general constitute a multi-f aceled and complex 
communication. Judges could detect these various facets but their * 
synthesis into an appropriate final categorization was not -an easy one. 

Error Analysis 

The implicit analysis of 'the test's p^llnpointed the potential 

complexity .of the item-person interactions. On^ demonstration of sex 

.differefrces lies,, of" course, in th^roportions selecting the correct " 

answer to questions. But .there a^rl^^^ferences in the selection of i 

the incorrect responses, also, and these can be a me^aningful potential 

soiirce of content relationships. The pioneer study^ of, racial and ethni 

differences by Davis and Eels *(i950), for examRjLe, considered these 

differences. Tables 28 and'..29 feport such item comparisons. 

. ^Item 5 iu STEP Reading 2A,*Part I is of interest in regard to its 

erroy distribution, -The' item stimulus . reads : ^ 

"The other -women thoughf Jlrs. Watson was being " 
extremely pretentious'* when she wore her nev.mink ' / 
coat to the ball 'game." ^ 

The sexes do not differ greatly on this item in*terms of level of 

success. The proportion of males succeeding is .921. The proportion o: 

females is .927, Howeyer, as '-Table 29 reveals, tSiere.are differences 

in the relative attraQtiV^e^s of th^ Varit^us misleads to this, item. - 

Pifty-five percent of the Vrong-answering females select response A, 
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versus'. 4'5% of the erfoneous males. Concomitantly, 27% of the. males 



prefer response 1, versus 13%. of females. 

, The content ^of^these *choi<:es is of inte'i^est.' Response 4., favored 
by females, is "uninformed;" response 1^ favored by males, is "unfair." 
On the surface «it would seem that there "could be complex ways in which . 
Hhe different toowljedges ^and frames ,of re£erenQ/e of §he sexes -are being 
brought, to^b^ar. . ^ 

Item 4, respbnse 4, is a wrong answer chosen twice -as -of ten J>y 
wrong-answering males (28%) as >by wrong- answering femstles (14^). Again, 
^the^verall success of vjt4\e/ sexes on" the item is very similar: .851 for 
males, .858 for females?**'"" ' ■ " • "^i 

Item 4 has the following stimulus:' „ . - o , 

In spite of the obvious y^wns and Vacant looks, on > 
the faces of his audience, the spealcei; launched Into 
anotKer interminable stbiy." \. ' 

Response 4 characterizes the speaker ♦as "an expert." What character- 

istics of ^ males or females could determine such results? 'Is the speakei;^- 

seen as expert more frequently by males perhaps because he is male, or - 

because he is boring? Both ^factors, of course, could operate. 



questions should be asked'and answered in a context of significMt 



differences. However, wlifile therp^ are several wAys of^ evaluating such . 
difference^ ^statistically, (CKi-square -comparisons of wryOng answer . • 
distributions, for example,) a difficulty with stieh analyses foT^the- *#- 

. V " ■ • ■ •• ■ ' i 

evaluation of 'content factors ia|||he fact that the total, Chirsquare ' 

^.-^ ...... 

obscures the ^contribution of the dndividju^ cell. Further, the level . ^ . 
of statistical sigi>if icance is a function^of the size of the SOTpl^'* , 
As noted In d number of earlier studies, it is often moire important to ' 

/ ' ■ ■ ■' . A . ■ - - - . ■ 
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responses as th^maln causal, fa.<jtor is impr&usible. The items as a 



whple provide a complex context which determines the interpreta^on or 
the options so that sex differences in response to' v^otds like "uninformed 

■ ' ^ * / , " • 

or "unfair" are made operative only in this context. 

The (demonstration ^f a fairly h;igh^frequency of appreciable differ-^ 
ences between the sexes in the selectfon gf wrong answer istopt ions 
contributes to the understanding of ±tS^ content *fap tors they relate , 
to^response. Sex diiEferences sho\ild not be seen as arising from the* * ^ 
stem only, or associated material, but ^rom' the item as a whole. 

But the study of tnese ^ex differences cannot obscure the general 1 
equivalence of the sexes. Of the sixty reading items studied, there 
were only fo'ur.dases where the tnost popular jjes^ponse for . wrong- answering 

males failed to be the most popular -response for wrong-answering females, 

. • ^ ^ • • A . . ^ ' 

only six cases whe;4fe the lea.st^popular options were different. The ^ 

fundamental picture is' one of great similarity b^etween the^sex^^ in the 

response to wi^itten language regardless of content. * \ 

There are theoretical implications from this type of error analysis, 

for an understanding oi--item difficulty; implications which ^must be - 

examined in vi^w of the number of sJrudies which have.fopused on coii^ar7 

isons of item difficulties, between the sexes. Jt might be conjectured, 

for example, that' observed .differences between *the . sexes in item 

» * • ' _ * 

diffictflty could be a consequent^q-f differences in the attractiveness 

of distracters. V .There i^ little fonnal •understanding of the role of ^ 

•• 1 "' • * ■ 

distr^ters in governing item difficulty: The practice* of item writing, 
as reflected .'in published guidelines reflects the belief .that widely 
remote distfacters do not mislead.t^nd henc^ faci^tate a ,r^sp6ndent's ^ 



correct response^ simply by reducing'*^the domain of ' possible correct answers 
If tl^s reasoning .is fotllbwed, differences in distracter att^ractiveness 
];could theoretically account for differfnces iij item difficXiity* The 
fundamental question' he^e considered, of course, is whether or ^ndt the 
choice of distracters is a* sensitive- element in the preparation of 
unbi^ed test material. Is there evidences that the characteristics of . 
distractefs govern^ or contribute to sex *dif f erences in the level of 

* . - .1 - . ■ 

success? Because the study can only /infer resp'onden^t thought process- 
during item response, no definitive answer can be given. In many cases . 
the overall difference in item success could be reduced t9 zero if ,,the 
errors b]j one sex on a given distracter were reduced to*" tlie' level -of 
the other sex and the level of re^orise to '.the correct answer incremented. 
But this model^ seems unjustified. ^ It suggests thaKfttie^power o£ a ' 
distracter. Mes in its capacity to induce in the respondent a mistaken 
conviction 'that this is t|!ie correct response. This might be c6nsidered 
a casie in whicftv^if fere^es Sz^i -peiiforJhan^ on distracter^ ' a're related 
to differences between the s^x:e^ in misinformation. v 

More likely^ however, it is ifot totally misinformation that governs 



• resporisre, nor does the "respondent have strong convictions ^bou<: the 

■ ■ . ' . • ' ■ 1 

answer. Instead, theitf^F^ndent has a state of uncer^inty as to the 
correct answer, and* the relative powers of the distracters lie iXtheia^ 
attrac't-^veness to these uncertain candidates* 

Ohe cannot rule otit, therefore,, the possibility that true sex 
equality in level of sucqess on an item might'be obscu^efl by an apparent 
inequality which derived fi-om dif/erei\ces among distracters* in their 
attractiveness to uncertain^pespondents. It seems mor^ likely, however^ 




that while the content of distractersUjifluences the choirs of wrong ♦ 
answer clif ferently for the sexes in some ca^es, it does/not constitute, 
the major .fac'to.r. 

^ Sximmary Discussion \ ' . 

The major focus of this stud^^was on the relation of sex differences 
in item difficulty^ to "content ^f^K^ors in tests. The majo^ ^findings ' 
would appear to be: • ^ , 

1. A significant tendency for females to do relatively better on 
items which contained femaley references • This w^ks established 
by correlations with tQtal female references across .all<ttems 
and by a regression analyses which focused only' bxi items with 

- li ' 

male and female references. Only Grade ^-\f inding^^departed 
from thia pattern. ' . - 

2. 'A sighi'-f icaijt tendency for both males and females , to do some- 
what -less well, on iteins which had greater jiumber^i^^f references. 

3. Consistency dn these'pattems regardless of whether the content 
variable studied was the total numt^er of references ^or the 
Xotal number of actors. ^The -total number of rol^s was a some- 
wh^ differ:fent, much Jess frequeit^^ly ap§(|icabl6 variable/ 

^ * whose pattern of correlation was somewhat more difficult to 



^ interpret. 

6 

4. F^w^ref erences to peopl^ in tests other than'^readi^g 'or 
vocabularv^ 

rj ' /■ ■ 

L/ 5% A tjendency for 

r •'■ • . - <^ . 

^ correlated with the Not Reached variable, suggestihg that 
Items witsh more references, pccur later in /Since it 



the number of refer^ndes to be significantly 



' . -82- ^' 
. . ■ ' ^ f 

is a dommon test development, practice to place harder items* / 

I 

later the* test, this finding is corJgruent with the finding! 

. 7 * 

on total references and difficulty. " - * 

* ^ *• 

6. Items with more than average references do. jiot differentiate 

... . - ,, * ''• N 

ability levels* within the groups, either males or ^females , 
quite as sharply as other items/ This could be simply a 
reflection of.^heir greater difficulty, for biserial indices 
tend to run^'i^er for harder ^tems-. * ' ' 

7. Thre sexes are highly similar jon such -noncontent factors as 

-A ' . ^ • - • 

^ omitting or rate-6f-work. Such factors do not explain any 



of the observed content-difficulty relat-ionshipd^. * 
.jr 8. Judges ratings ii; an- implicit analysis proved less effective. 



at predd^cting sex difference than did the explicit measures 
of content. In general, the implicit^ analyses reflected 
quite different fiptors in item content. , ' ^ . • , 

9. , Examination of wrong answer patterns revealed a small number o' 
«^ interesting examples 'of sex differences apparently relafced'^to 
the content of specific wron^ ^swers and t*o sex differences 
in misinformation. Such fittings could not diminish the over-^ 
all impression of substantival agreement between the sexes, nor 
..were the differences in wrong Answers suf f-iei^ntly large to ^ 
explain the differences in item success 



if 



As in any complex. .area of^,tudy, the investigation stimulates many 
new questions as it ari^wers some oliJ ones. It establishes sign^-ficant , 
but modest/correlations between Ifex differences ^in success on items and 
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the content of such items. While- tb^ level of these relationships^ and 
■ the low overall frequency of explicit references to people would not 
make item content 'th^ prime so^'ce. of sex differences, the finding is an 
interesting one. Since' this study confirms the finding of Ti^le that 
male references vastly -outnumbfe-r female references in tests, the deHon- ^ 
stration tjiat female references modulate male-female difficulty differences 
has si^ificant implications for test development^ practice - . * 

Several limitations of this analysis should be discussed. ' First, • 
there is the use of content analytic totals.' These totals summed aJLl ^ 
referei^ces to individuals and to groups, "obscuring potential differences 
in the impact of thQ^e two levels ^ Second, there 'is the redundarrcy in ' 
the concepts of references, actbrs and roles. Th^s redundancy, while it 
can assist in the interpretation of patterns of coef fic^rents, may also 

ke it more difficult. Thirdly^ the .pfl'ilcipal analysiW:onsiders only ' 
-fail differences in item response, ignoring the possible distinctions 
in wrong answer responding. ^ v 

Some suggestions for future research Voul^ be:* ^ 
' 1. To conduct studies of item^solution processes as ^^se 
differ for the sexes.- Little work of this- tyTffe has b 




een 




done. Connolly and Wantman (1964) offer the best example 
to date^ Such a study^ would clarify and expand many • ■ • 
elements of wha^ is covered |;iere.\ 

To conduct studies^ of "qu^si-parallel" farms of/6fsts^ 



forms parallel in every/sense but with sex references 

— — 

varied in *sys temple ways . Such^ studies woyld provide^ 

i i ' \ ^ ' ' > " r • 

s4 Iclearer tests of the impact of content on difficulty. 



3. To pursue the exploration c^f* content subareas and $ex differ- ' 
ences, invert ingy the process used in the present study and 
, seeking to establish the content characteristics of large- 
, ' sex -dif f ei!^nce Items. This Approach characterized the ^ 
earlier ap{)roach ^f-Coffman. It 'may reveal test character-'" 
,^ ,istics othe^r than content relatejj to sex differences in 
success. . . ' 

It Is, intone sense, reassuriiig *to find the low level of statlsbix^al 
effects associated with content effects in this stSdy. However, this 
finding must not^'be translated into a con^rlusion'^ that* ajttentlon to ' . 
balanced sex referenc^es~ in tests is inconsequential. #lsts are an impor- 
tant sampling of culturally signif i'catjt behaviors. ,,The. conditiojis of ' 



J. 



this sampling should -reflects the values of the culture. The patterns ' 
of sex di^erentiation are in transition, and the older practices of 
.J^more frequent references to males, of stereotypical sex-role associations. 



^and of sex ahd^s'tatus linking sho*uld be abandoned. Bias in tests is 
not only demonstrated, by statistical tests of the scores, but by the 
relative incidence of references to males and females. 
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* 

0 AManual^for Identifying Sex Bias Explicit 
^ ^ in the Cbntent of Test Items* 



Sex stereotyping in both the language anS dontent of achievement tes^ts has / 
•. been ndted by a numhgr; of researchers, including Coffman (1961) bonlon^*971) , 

Lockheed (1973) and-^ittle (1973). S3^ stereotyping has been-describedl^q)licitly 
^as tUe absolute incidence of male and female references' in the tests, as, the rela- 
tive status of males and females In the tests,, as -the incidence of s ex-stereo typedTr 
roles in the tests, and implicitly as the sex-stereotyped content of the ^est^» ^' *. 
In general, it 'has been found that the language of tests favor^ale-reference.s x>ver 
female references, depicts malls in higher stat;us roles than f^msLles, and , describes 
males and females as stereotygd^l in botlj roles ^^ria^^characteristics?^- , * ' , ^ 

What the effect of sucH stereotyping has been on the performance of/ test- 
takers is not known." Suggestions of the effect of test content on peff^fcmance^ 
were found by both Coffman and Pqnlon,^o report tffat test items "di'sci^^^nating 
between males and females statistically were found t;o have substantially 4i.jEferent 
conteAt. - ^ ' ' * • / ^ ' s 

' ' ' . ' '% \ 

The purpose of the present manual is'' to describe tests in such' a way that ' 
^the -explicit .stereotyping of language\ .roles and sex dmbalairce in cont^t may be 

related to indiyi'dual test perf ormance* In order to relate sex stei^lj^^ing in 

tests to the -actual test performance* of males, and females, it is ne^l^y 'to ^ 
' select a unit of analysis most ^appropriate for this task. We have determined 

that the -appf op^riate unit of analysis for this purpose, ,ia the test item itself. 

Although the irianual was prqduc&d for a particular study, it can serve as. a means . 

of determining sex imbalance explicit^'in the content of any test. 

A t^st Item is defined(as having three bas'lTc comp/onents: "l) the stimulus, 
--which^pravides .the background for the question, 2) the st^m, which asks_ the 

question^ and 3) the responsfe(s), one of which isUceyed as "coijrect." These item' 
•* iharact^riiptics may be analyzed set)arately to determine theix independent' effect 
.-on thgft:est' taker's* performance. . ''^ ^ 

^p'expligit content anal^sid^ of the items ^ this llftual focuses on three 
conceiitii^lly -distincjt' but undbubtedly correlated indicators of sex ^imbalance; 
These a^e: 1) the telative ba^iLance of male References to feptiale references,. 
2^ the relative balapce of maleS to females, and 3) tKe ^oles^attributed to -males 
and females. ^ ' • ' ' , . 

Ey fociisin^. .actors anJ roles in . this jHRtent arialysis, we are acknow-- 
ledging pur sociolog-ical bias. This aspect ox^ipfe analysis entails asking^ whether 
the tast taker^ia performance on the test is related to the extent to whi^ the. 
, test taker /"perceives that the content' of the test i^s appropriate for h^m or fi^'r. 
^Th^t i^s, we. are asking whether *^role typificatidn for males and female^ affects 
test perfprmance. llble typification may be communicsitfed. by the-ialance of\»males f 
and'ffemal^Sj by the^balance of '-male and femare referenced, and by the* actual 
p'ortrayalrof males 'and females in various roles. ' ^ W - 



Support ,'for the developnjent of this m^nua^ came f rom^a'^rant to Educational 
Testing ^ervice from the Na.tional Institute of Education (NIE-G- 74-0008) ^entit'JLey 
"Performdnce Goj^quences of Sex Bigs in. the -Content of Mdjor Achi^fement Test 
Bdtteriei." . ' 
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Description of 



the Mantili ./ 



The contents' of - this manual in^ude^fpuf sectlons't (1) Coding Format, 
(2>-Coding "Procedures, (3) T5e«dLs^<^fj:ules , arid '(^) Use of the Coding and 
Annotation She'ets. 



ERIC 



Coding Format • - ' ' ' . • ^ ' 

the Coding^ Format section describes the co'dable coihpoiients of each test item. 
As mentioned preyiously, these components are the stimulus, "--the stem,^ and the. 
''responses. The stimulus, component of an item is the background information needed 
ta answer the question, Xt ma^ be stated directly In the question or assumed to 
be known by the test taker i>- The stem is" the component 6f an item which demands 
«a Response. ,The responses" a*re the possible answers to tlie , question. Methods for 
identifying each component of the item are explained in this section. 

Coding Procedures • . 

The Qoding 'Procedures section outlines tlie methods used to datermine se^ im- 
balance in item content., > Sex imbaLance may be defined as the relative balance' of 
male and female actcJrs, references to male and female actors, inale and female 
rolfes, 'and .the relative* status of male^-^and female^ roj-es . 

•.,In order*' to do^termine the relative balance of m^lies and females, the 6ex of 
Che sac tors must be identified.. Four- procedures :are used: " \ 

1. The noun is • inherently sex-blinked,^ e.g. , mother, » ^ 
father, sister, brother. ' ^ « - 

2» The noun is "found to have a sex specific definition 
* . in the dictionary*", e.g., ^ - /T^ 

ballerina 1. a' principal female 
dancer in a ballet conlpany,*2. an^, 
female ballet dancer 




le noun is a definite female or male name," e.g. 
Jill, Maty. ' ""^ ^ 

/"rtie noun^ has a male lor fe^le pronoun which refers ^ . 

•to/it, e.g., Pat went to her class. ' ^ * 

The actor (s) /in the item, either single intiividuals or 'a group of individuals, are ^ 
th^n counted/. In caSe^ where sex cannot, be assigned, a neut^.a^ category is used. 

'\ In' ordir to determine the relative balance of male to^, female references, th^ 
number of aJbtors plus other words SfUqfi'as pronouns which refer to them in the it;iem 
)at:e counteq. ' ' * * ' 




The dictionary selected for use for d,eterminir\g sex specific definitions wa; 

The Random House Dictionary, of the English Language , 1967. '.^A recently p^bl joshed \ 

^dlctionaiiy was chosen on the^ belief that "it would be lesd likely to contain sex-/ 
stereotyped def iiditions. 
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t '*' Words which show vocations, avocations*, or special functions of people (f9r 
example, doctor, ^mother) are coded as roles. R^les are not inferred from tHe 
descriptions of individual behavior. Eor example ^ the role of "househusband" is' 
not inf e^crea^f rom the sentence, "He cleaned the hodsB an'd fixed dinner".- ,The' 
identification' of particular roles as fepiale, male, or'nfeiutral was decided by 
the peir.centage of /emales and males found actively ^engaged in that r^le as docu- 
mented by the C)ccupa,tipnal Characteristics ,197tf Census of Population and other 
sources. When •8^% or more of the individuals engaged in att occupat:ioji_were one x 
sex, the pccupa^£ion was defined as a sex-typed role. Other occupatigns were 
classified as neutral roles. Historical consideration 'of roles w^s handled by 
general knowledge. If an item involved a rolfe that was generalfy known 'as ^ 
Restricted to one sex (e.g., knights, the congressmen of 1800)>, it .was coded as a 
/sex-typed role. . • ■ ' " , ' t» 

/ In order to assess status, both mal^s and females need to be present in an 
item. iVssessing the status" of roles in an item focuses "on the sex of the-acto'r 
h(^lding^hq| mo^t superior rolie. The, status of males ^nd fejnales in an item can 
be equal i female superior, or male superior. 



Decision Hules 



/ 



The Decisibn 



^ iSli^es section provides directions/for 'codiifg special situations. 

Particular problems have been found- in identifying^actors or groups, dealing with 
the structifre of the English language,, treating >mithropomorphi zing of animals'^, and . 
handling ambiguous roles. In the. manual these/problems have been organized under, 
the fo'llowing head^ings .which relate to the c^ing procedure: Rules for Identifying' 
the Sex of Distinct Entities and Referenc^V Rules for Counting Distinct Entities 
and Refer<ences, ' Rules for 1)esignating Ra]^, and Hiscell-anebus Rules. 

>! ' ' - / - ./ 

The yse of the Coding, and. Afin6taJ:ion^>^heetjS ' /' 

Special rules and procedures 'which a^e/specific to the study '"Performance 
Consequences of Sex Bias in T^^t."^ I teips"^r4 covered in this section. -It describes 
how th| coding sheet is^.used,/^here on t^e coding* sheet the content lof tfi^ pest 
questions are coded, and how irregularities and subjective coder decisions are 
documensted. , * . ■ 




CODING". FORMAT 

^ ■ .1 '] 

Test questions are coded by separating each item into stimulus ^background 
information'^ven or assumed known by the reader), stem (question wjiich directly 
t;riggers a response) , and f es|)on^e .sections, tor example:*. 



sf iinulus T 
Stem ^ 



/ 



-^-^*^he* deep-sea diver weqt -^own twenty fathoms. 



-9^A fathota is. equal to 



response 1- 
response'^s? " 
response 3"- 
response 4 • 

I 



60 feet ' . 
6 feert ' 



-^d. 12 yards 
>D. U2 feet ^ [' 



. Format I. Most test questions are Easily, ^lalyzefl^ Into the' three sections listed 
a^ove'and are frequently found ±jx the, following form: 



Da,le was five feet tall*- (stimulus) 
.A^foot is eqiiaX to (stem) 



A. 2 Inches (response; 1) 

B. 6 inches (response 2) 

C. ^ 9 inches' (respofise 3) 

D. 12 irithes Oespoftse 4) 



Format !!• The stimulus, ste&. and response sections may not necessarily ]>e listed 
in that order. For exa5q)lei/ ' , ' . s 



X 



The following ^sentence refers to which^o^f ^th,e^ foil owing 
meanings of .the word "desierts"? (stem)-^ ' 

• / ' > ' * » ' 

He received his just deSerts when his job was given to 
someone else. (stimulusO v\ ^ . * 



A. reward/ ^ (response 1) 

B. wast?e/ (response 2) 
forstafke ^(response 3) 

D. punishment (response 4)*\ 



'Format III, 



Frequently a 'reading pas]^age> table, feraph^ map or picture is followed 
.by a. group of items referring ^ack to it. In such cases, the^passage, 
table, /graph, etc., is coded as the st:imulus and each new <juest^.or} ' 
referring to it Is coded as'.-a stem of that initial stimulus^ > . 

For example r . , , - 



0 threat&-of *Hell~ and Hopes of Paradisel , ' ' * 
One thing at least is certain — • This life flies; 

One thing is certain and, the rest is Li^s; 
The Flower \hat onqe has blpwn for ever- dies, (stimulus) 

The fourth line restJ^tes the second :yLtie;.(st6m) \ • 

A. literally (response \\ K \ 

. B. structurally ^ " (response 2) • . ' 

* C. * onomatopebtically (response 3)' V-- 

D. paradoxic^ally (response 4) 

E. figuratively ^ (response .5) . ' ' 



Format IV. 



In the fourth line "blown" means (stem) 



A. blown up (response 1) 

B> blown away (response 2) 

C. bloomed (i^esponse 3) 

D. died ^ , (rrfsponse^ 4)' 
E'. been planted (response 5) 



4 ^ 



some items the stimulus and- response material may be t^he same* and- 
therefore double-coded • For example : 




Whi^h of the following are parts of a flowfer? (stem) 



1. ovar}^ * & 

2. style Y (stimulus> 

3. pistilj 



\ 



\ 



A. 1 and 2 only (response 1) '[Cpd'e as if reading text ^o,f 1 and 2] 
,B. 2 and J^only -(responsi 2) '[.Code as if reading text of 2 and 2] 
Cj ^1 and 3 only (response. 3) [Code as if reading .text oOi and 3] 



Format VJ 



Format VI. 
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Many items' €0 not cotifain three sections or do*. jiot. refer back^tc^an 
initial reading passage, ' table, • graph, etc. lriv^.^uch' items, the * 
stimulus is background information not given- to the r^ak^rs and Is, 
not coded.^ Code the. question asked as^ the /stem and code the're- 
sponses as usual. For examp"l.e:= \ * . 



Which of the following is notj one of tjie uses- of 
Qhlorophyll? -(stem) 

■^A./% used to produce carbohydratfes (response 1) \% , 

a. used ^s a. disinfectant (response 2) , 

C. -* used ^8 a dye: - ' (resppnse 3) 

D. used as a deodorant (response 4) ^ ' 

Often in' English expression and writing items, passages are pre- 
sented which have tKe^ response choices incorp,gr^ted into them. ^ 
In such cases, coda tSKe entire paasage as ' theJ^sbCmuius, inclu4in'g 
the .underlined sections designating tesponse c^ipices.* Then code 
the underlined sections individually- as response choices. (In 
such cases the stem is' implicit and is not .cbdfe* since no direct 
question i^ asked of the te9tvtaker.) ' . ' . 




. Example <'l: John grabbed the gall from^his friend than went 

; A B . - _ ' C 

. oiit to play. No error^ (s1>imulus) \ 

. . . D - *. 



Eormat VI J. 



Example 2: 



A = (response 1) ^ 1 

B «^ ^(response 2) ^ ^ 

C = (response 3) ; . ^ " . \ 

D = -Response A)? "^ ' 

In ancient ^ Greek , large cofumns were a dominant 



'tja'rt. t)f the architecture. 



No. er.ror . , (stimulus) 
D 



A - (response 3,) . ' 
B = (response 2) ^ ' ' 
a = (rd^poiise J) - ^ . 

D = (respbnse 4) 

In spelling items, the test taker is ofteft presented with alter- 
native word spellings of a word^ to choose from. In ' such *qases, 
^simplj^ cpde each choice as a' respOiisel For example': 



,B-. 



D. 



scend' . (response, 1) 
send . '^j*. (response 2) 
scent ^ (response 3) 
no error (re3p6nse 4) 



/ - 4 - 



'CODING' P-R6CEDURES 



- * - . \ ' " . ' . . ' ^ ' ^ ^ 

Cd^ittg* Procedures diitline .the method^ used in coding -th^' content of test 
'items. It is divided into seven s'eations: " general information, ca^tegory types, 
counting* distinct entitie^^ountingj references, assigilStig^ toles 5^ counting 
roleife' and determining sta^s.^ E^gn of the thirt^ynine holding categories is 
identified by name ai^d 'mnemoJjti'li found on the coding sheet. CThe mnemonic is in 
capital "letters. ) "JFor further ceferen-ce, please see the Sample Coding Sheet 
on page 24. - , ♦* 

Sectiqn I: General Information* _ • - ■ '■ 

m ■ 

A.-^ Test Idetitif ication ID 

• « ' 'j . ^ - . * 

The number which identifies tlje. test heing coded is ' ^ 
tallied ifi thxp category The codje for each "test included" in the 
s • 'present study m as follows: 



="CAT (Calif arnia Achievement Test/)' y^^.^ 
= -ITBS ^(lowa Test of Basic Skills) P 
, 3 = MAX (Metropolitan AchieVeYnent Test) y ' 

4 = STEP (Sequential Tests, of Educational Pmgjress) 



r 



B« Section of Tefet \. . . - \. , SECTION ' 

%^ The .nuniber which identifies which part of: the test b^ing ^ * 

J . • coded,^^ tallied in this categpry; ""The code for e^ch part is' 
"^W*/"^.' as follows: - ' ' , ' , ' 

. : ' ^ % ''2^ Part ,11 ■ " • . ' 

^ ^ 3*= P^rt III 

^ ^ = Part IV ' • ' . ' 

j ^ - * -(etcO^: ' \ ' . ^ ^ ^ 

C. Section Content ^ ' , \ ' . CONTENT . 

The kind of content in the section' of the test being ^ f 

tallied^ in this category. The codes for the content of the 
sections included in- the present study are .as follows*: ^ 



1 = Reading^ Comprehension , ^ * 

2 = Language Arts (writing,, grammar/ spelling) 



3 = Mathematics Problems 

4 =. Social Studies- 

5 Science ' 

6 = Work-Study Skills 
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D. Item Identification 



The number of the' test question or test item is 
coded-^in this c2tegory/ The codes. for the items or . 
questions are' as follows: 

* 001^ "test question 1 
• ' 002 = test question 2 

(etc.) ^ 



Section II: Category Types 



ITEM ID 



CAT EG 



The coder makes the decision as to whether the item contains 
primarily (1) text or verbal, (2) pictorial., or (3) numeric or symbolic 
material and codes^the type, number according to the decisioji. Coder de-* 
Cision is based on the relative number of words, numbers," or pictures in 
an item, and on consideration of 'th^ir function. The codes for the 
category types are as follows: * . ^ ^ ^ * ' 



' 1 = primarily text/verbal ' ' \ 

2 = primarily pictorial (includes maps, charts, 
'/ ^ ' ^ cartoons with little or 

' ' ' - no textlial material) 

' • ' 3 = primarily numeric/gymboll^ v. 

4 = combinaticfn of two or more of the^ above 

Example i: Mary, Mary quite contrary; how does 
f your garden grow: 

Code: CATEG =1 « ' 



Example ;2: . 




Example 3 ; 



Code: . CATEG = 2 
5 4120 + 4 CO^ •= 



Code*- CATEG = 3 



/ ' 

.Bxanrple 4: Ingject- Life Spap (da^s) 

-v ' — ■• ^ 

. • ^ ^ fly , • '100. 

mosoKiito ' \ 125' m 
butterfl'y 175 



- S X * 
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C ... ' Code : CATEG= 4 



If the' item does* not contain one or more of »tKe aeven possibly components 
(stfimulus^, stem, respbnse 1, response 2, , response ^, response 4, "Response 5) 
of the usual fonqgts, a zetp is coded in the CATEG category to show* that thej 
particular component is missing. * ',!',[*' . " 

. « Example: WiiicK is the color of the suW? • 

* * * ^ ' * — ' 

^ A. blue ' " ' . , . 

B.'^^rown ^ ' ' ' ^ , 

/ C. yellow ' ' 

D. purple ^ ' ► 

» . * • • 

Code Stimulus: CATEG « 0 
Code Stem: CATEG = 1 ^ ' ' 

. Code each response : (A-D) CATEG '= 1 

Code fifth response^: CATEG^ = 0 . 



Counting People ' ^ ' • 

A. Individuals: . Female,. Male ^ Neutral -^m^ liAL^ NUT 

The number of distinct female, maife', and neutral people in thfe 
item is tallied here. A flioun which refers to people is coded as V 
female or male:' ;(1) if it is inherently sex linked /(see Table A, page • 
26), (2) if it' ft found. tp have'a sex specific definition in the dic- 
tionary (see T^ablfe A, page 2^), (3) if ^it is^a definite male or female 
name, (4)' if it has ^ male or female pronbun which refers to it. When 
these conditions are not met, the noun is coded. as neutral. Personal . 
pronouns' (such as -I, we, you, she,' he, etc.) are counted as distinct ' 
persons in the item when they are not references to a specific personal 
noun . ■ *^ • - ' * , 

. Example l:^The king and, queen ruled well. 

. Code:* FEM = 1 (queenTV \ ' 

« - ; ' llAL = i (k^ng) J f herently seoc-linked 

Example 2: She Iboked like, her moiher. 

, . , ' .Code: PEM = 2 (she,- mother)^ inherently sex-linkedj 
'^xam^le 3:j^he poetess was be coming> famous, ' ^ ^ 

Code:, FEM = 1 (poetessf^ sex specific definition 
Example'4: The doctor grabbed her 'bag 'and rushed out the doot'..\ 
Code: -FEM = 1 (dbctor) f female pronoun referent 

" . • • - r y ' - • ♦ » ^ 

Example 5: ^he, nurse worked the night shift. - a 

.^ode: NUP= 1 (nurse)" A none of the 4 conditions 
^ above. [Note th^because 

. ' . ^ V most Qurses are 'female, this 

would be cod«d as. a female 



.role.] ' L 

r 



GPNUT 



Groups: Female, Male,) Neutral (or mixed-sexO GP^, GPMAL, 

- The number ^of distinct female, male or freutfral (or mixed-sex) 
groups^ of people are tallied here. . The same pr6cedures apply to coding ' 
groups as female, male, or neutral groups as to the c^ing of individuals . 
in the previous categories. Both plural aftd co^llect'ive nouns are coded 
as groups. A mixed-sex group is codeS as GPNUT., 

Example 1; The nuns went to^vespers. * . » ^ 

Codef GPFEM = 1 /nuns) > inherently sex-linked 

Example 2: The 3 boy scout troc/ps met after school* 

. Code: GPMAL = 3 (3 boy scout trroops)^ , inherently sex-linked 

r 

' . . ■ ^ .106 - ' . . ' ^ 
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Example 3: Xhe team had a lot of spirit, J; .none of the 4 conditions 

Cocje: GPNUT,= «1 (teai) • , ^ \ 

Example 4: Congratulat-ions to the graduating students! 

Code: GPNUT = 1 (students) y mixed-sex group 



0, 



Section iV, C6unting Reference's ' * » . ^ , r—.-Ty \ 

' ' * ^ ' ' . - • ^ ' ^ 1 

•A. References to Individuals^: Female, Male, Neutral * FEMREF, 'MALREF^ NUTREB. ^ 

• . • The number of references to female, male and' nfeutr^l fi\dlViduals 
in"* the itfem, including both nouns and'pronauns, are tallied here- ^ . , 



Example :}.:\ She loved her new car. 

^ Code:- FEMREF' = 2 (she, her)^ : 
" Example 2: John and Tom went .'to 'the show^ 



;^sex-linkeci pronouns 



Code: MALREF = 2 (Jo^n, Tom)^,, male n^es L 



. i 



Example* 3*: 



J^ck had his own car and Jill 
had hers. * 



Coder FEMREF = 2 (Jill> 
MALREF =2 (1 



ill>'heri3^ t 
acky his)2y 



female name and pronoun 
a^e name an?l pronoun' 



-Example 4: Pat was nominated to riin for office. ^ ^ > ♦ • 

< * • * ' ' '\ 

Code: NUTREF = ,1* (Pat) V .name which could either; mle or female 



1 



B. ReS^rences to Groups: .Female, Male, Neutral GPFEMREF, GPMALREF, ; 
' . . \ . ' ' ^ ..GPNUTREF 

The numbey of references to female', male and nfeutral (or 
mixed-sex) groups in- the item are tallied here. tShen plural pro- 
nouns ref ^ to groups of different sex<^s, tM reference is coded 
^^.^ as GPNUTRB^. (GPNUTREF Is a neutral reference or a mixed-sex 
reference) • ^ 4 



Example 1: The crowd expanded until it poured into the street. 

. • Code* .GPNUTREF = 2 (crow^ it) .. U • ' ' ' 



ERJC 



JBxample 2: The amy was composed of brave men. 

rode:- GPMAMJEF = '2*(army, men) 



Example 3: The ^Ballet" class'' had many students. ' ' ^' • ^ /C* 
. .r ' ^ ^* \ • - \ - » , * / 

Code; GPNUTRE? = 2 (class, students) 

Example' 4: ^'Jack and Jill had their problems.^ • ' . ^ 

Code.- GPNUTRPF = 1 (their)^ ^ 

Assigning Roles ' * • c * . » , r . t • 

• , ' • V ' ' ^ 

(Foi:'the*preselit study', a role is defined as a vocatiou, an avocation/ 

,or a special human ^unctipri.) ^ - ' • . " ^ * " „ ' • 

-/ ' ' » * ^ : * / 

A^ 'Individuals'^ ih Roles: [ Female Roles FEMROLF, FEMROLM, IFEMROLN 

The number of,timea tha^' 4 female, male^qr neutral individual' appears 
in a fem^ role is tal^Lied herfe. Decisions^ as to whethef the. ro,le in the 
item'is a female role can be aided by Table B Jsee page^27).' When a fe- 
male individual has a female role it is tallied in the* FEMROLF, category ; 
If a male individual has a female role, it is t^llie^ in the FEMROLM 
categdry.^If a jieutral individual' hag a^emale: role*, it is tallied in - 
the FEMROLN category.. - ' , ' • 

Exkmple 1: Sally was'Hiired .as-4' nurse at a hospital nearby. ' - • . % 

- ^ Code: FEMROLF = 1 (ncirse) • ' ' 
\ • ' < • . " ^ \^ ' 

Example 2: Being a pro^^perous hairdresser, he'owneo^his own>shop. T 

Code: ^;FEfffiDLM = 1- (hairdress#r) ^ ' 

Example 3: A flight at tdndanr should not be^ too fat. - \ 

C6de: . FEMROLN = 1 (flight attendant) ' ' ' - ' ' 

B. . Individuals in RoleS: Male Roles MALROLF,' MALROLM, ^IALRPLNl 

The. number of 'times that a female, male or neiitral individual 
appears ►in -a mal'e rdle'ls tallied here, decisions. as to vhether 
the role In the item is a male . role' can be aided by TaEJe B ("see page .27). 
When ^ female individual has a-lfi3le role, "it is tallied in the MAI.EOLF ' 
category.' When a. male individual has a male- rb3.e, it is ^ tallied in ; 
the MALRbLM category.^ When a neutral individual has a "iftale role, it is 
tallied: in the MALROLN oategory^. ' * • . , 

Example -1: As -a judge, she worked long hours. . . . •, 

Co'de:"., MAf^ROLF = 1 (judge) " ' ' - 



Example 2: He^ became a minister,^ 

Code: MALROLM = 1 (minister) 

Example 3: Se'ieing ^ lawyer can be e;pensive, i 
Code: MMJIQLN = 1 (lawyer) 

C* Individuals in Roles: Neutral 'Roles 



NUTROLF> NUTROLM, NUTROLN 



The numbej of times that a female, male or nelitral individual • 
appears JLn a neutral role is tallied here. Decisions "^as to- whether 
the role in the item is. a neutral role can be aided by Table B (see 
page 27), If a decision cannot be made as to whether the role is ^ 
male* or female, it should be coded a's heutfral. When a female in- 
dividual has a neutral role, jLt is tallied in the Nin;ROLF^categ&ry. 
If a'male individual has 'a neutral role, it is tallied in the 
^NUTROLM category. If a neutral individual has a neutral role, it is 
Jtallied in .^tli^ NUTROLN ^^ategbry. . . . 

Example 1: Brenda becaW' ah author at 16, 

7 . ^ ' 

. Code: NUTROLF = 1 (author) - ^ . . 

Example«2: He enjoyed a long cai:;eer as a psychology ' 
professpr at a small college* 

Code: NUTROLM == 1 * (psychology profe3Sor) 

Example 3:: Aiv actor experiences an up and down career* - 




Code: NUTROLN|f 1 factor) 
D« Groups in Roles: Female. Boles 



FEMROLFG, FEMROLMB, FEMRO: 



The number of times that a female, male or neutral (or mixed-sex) 
group appears- in a female role^ is,, tallied here. Decisions as to 
whether the role in'.lfhe ±tev^^ a female role are aided by Table B 
(see page 27)* FeniaCle, male, or neutral (or mixea-sex) groups in a 
femkle role are tallied in>'the same manner as individuals in female 
roles • . ' 

Example, 1: The wives .held interfestin^ jobs* 

c Code:'^ FEMROLFG = l\(wive6) 

Example 2: The men. were eii^oyed as elementary school teachers ♦ 
Code: MALRDLMG f 1 '(men) \ ^ 



FEkROLMG-= 1 (elementary school teachers) 



Example 3: The nurses went to their class ^ . • ' 

Code: . FEMROLNG = 1 (nurses) 

fi. Groups in Roles: Male. Roles MALROLFG, MALROLMG, MALRQLN.G 

,^ The number of time§ that -a female, male or neutral (or- mixed-sex) . 
Vgroup appears in a male role is tallied here. Decisions as to whether 
the role'in the item is a male^role are aided by Table B (see pag6 27). 
Female, male, or tieutral (or mixed-»sex) groups in a male role are tallied 
in the same manner as individuals in male roles. ^ 

Example 1: As generals^ the women commanded respect throughout the 
ranks. ^ ' 

'Code: MALROLFG = 1 (generals) ^ FEMROLFG = 1 (wpmfen) 

Example 2: The men went -to classes to learn. how t;o be bartenders. 

Code: MALROLMG = 2 (men, bar4:enders), 
Example 3: The taxi cab drivers were on strike..- < 

Code: MALROLNG - 1 (taxi cab drivers) 

F. Groups in Roles: Neu-tral' Roles NUTROLFG, NUTROLMG, NOTROLNG 

The number of times that a female, ma nqutral (or mixed-^ex) 

group- appears in d neutral role is- here. Decisions as to whether .the , 
role of a group ±s a neutral role are aided by Table B (see page 27) . 
Female, male, or neutral (or mi^ed-sex groups) ^in a neutral role ar^ 
tallied in the same manner as individuals in 'neutral role^. 

Example 1:" The female editors of Mc(iaj.l's did an article on 
^ ^ women in publishing., \ ^ % 

Code: NUTROLFS .= 1 ^(female editors) FEMROLFG = 1^' (women) 

Example i: The brothers ^wdre kndVn as top paid aQtors. 

Code: NUTROLMG = 1 (actor^s) MALROLMG (brothers) 
f * ' . ' " . ' ^ ' . • . 

Example 3: The garage workers and gas station attendants wanted 
higher wages. 

Code: NUTROLNG.^ 2 (garage workers, gas station attendants) 

» ' ' f ^ - 

* ' ' * V 
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Counting Roles J ' ' 

A. jr^Total Number of Female Roles - .TOTFEMROt 

"u ■ . , : ^ 

The total number of different -female roles found in tfie 
it^m is^ poded In €iiis category. • ' 

/Example' 1: My mother is a receptionist part-time and a 
^ nugf^e part-time. 

Code: TOTFJIMROL. = 3^ (mother, receptionist, nursej 

Example ^2:' Ey.1 is d hairdresser and Ethel is a ^stewardess . 

, Code: TOTFEMROL = 2 (hairdresser, stewardess) 

B. Total Number of Male Roles > TOTMALROL 

The total jiumber of different maJ^e roles found in the 
item is coded in this category. , . ~ . 



Example 1: The = cab drivers and airplane pilots were on strike^. 

' Code: TOTMALROL = 2 (cab drivers, airplane pilots)- 

Example 2: Studying to be an economist or a physicist is hard. 

Code: TOTMALROL = 2 (economist, physicist) i 

C;- Total NuTfber 6t Roles ' TOTROL 

The total number "of the different roles, female,* and ipale 
and neutral, are coded in -this category . "If .ohl^T neutral roles 
ar^ found in the item, their total is coded in this 'category. 

E3caiiq)le 1: When my slsterNOT^oKs tip, ^he wants to be a 
doctor, a lawyer^ 'qr a musician. 



ERLC 



. Code: TOTROL = 4 (si^^r, doctor, lawyer, musician) > 

^Example 2: Once upon a , time there was a princess vho was "lonely. 

She tried to find the jester in^ the palace to cheer 
her up but the- jester was busy entertaining the knights 
and squires and princes so she stayed lonely. * . 

• -Code: TOTROL = 5 (princess, knights,* squires, 
* princes, jester) 

Example 3:^ The editors and reporters praised the 3cuipeor*s^ 

new work. * - ' • , y 

r -Code: ^OTfiOL = -3 (edi^idrs, reporters, sculptor) 



/ ' r 



s 1 
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Section Vlt 



Det ermin ing S t at us 



ST/ 



- The relativ^e status of female- and male ro/es in the it6m is . 
coded^ in this category. The item h^s. t<y have /female rol©s/and ma3.e roles 
present in order to. be' coded here. The/role/nierarphy is /coded by focusing 
on the sex of the person holding the nidst superior rolfe . //If the 
' item^ does not have both female and male present, it i's ' / 
coded as. "not applicable." The.codi^ig of Status ia'ias follows 



tus 



1 »= equal female and male stai! 

2 = female. status superior uf male status/ ^ 
3v^ ma^e ctatui fiuperi/r to yComaXa statu/ 
4 = not applicable 



Example 1: Bill 'and Mary are lawyers 

Code:' STATUS = 1 (lawyiirs) ^ 
Example 2; The mother, smiled p rpualy _at her .g</n. \ 



11 ' 

, Code.* STATUS » 2. (mother > I'son)^ 

Example 3: , The ^aught^r, refused^ to be married 

^ * ^ although her father pleaded Vith-her. 

Code: STATUS =* 3 (rather daughter) 



Example 4-: ' She can run faster jthan he can. 

Code: STATUS « 4 (no roles) 



Exanq>le 5: ChS* dpctor^ went hcraie, eariy. 



Code; Status 



(only on^ role^.. doctor) 



.ERIC 
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DECISION BULES 



Speci^l/situations which arise within the previous thixtyrnine categories are* 
presented - in tihis section, along with rules governing their coding. The Decision 
Rules or coding are divided into four seqtions: Identifying the ?ex of Distinct 
Entities artid References Counting Distinct Entities and' References , Designating 
; Rolea. a.ti<i Miscellaneous Rules • . * - 

Identifying the Sex ot Distirict Entities anH Ref ereitces^ , * ^' . 



Rule l.j ' If se3i is assigned to an .individual , or greup at any point, carry that 
, » sex through 'the mAtPHaV. * " — 



sex through -the materials. 
Exaii5)le;l: 'The dentist went to see his patient; 

.- Code: 'mAL = 1 (dentist) ' • 
, NUT = 1 (patient) 



MALREF = 2 (dentist, his) 
NUTREF = 1 (patient) 

MALROLM = 1 (dentist)' ' 
.miTRnT.N = 1 (patlentj 




A 

Rule £• When sex ^^.*assigned to an indivi<fual oo: groupl in an item, carry the sex 

through twT stem and respenses provided that t^^ sense »of personal identity 
. ±s coutiuued in the steii^ and reaiponses. • ' 



1 1 
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Exanq)le 1: / ^. 

Stimulus? Although the lawyer spoke forcefully, 
she did not sway the jury. 

*Stem: The lawyer can best be described^ 

, as which of the following? ■ ^ * 

Responses: "STlIhe lav^er is a* persuader. * 
^ B» ^The lawy^i; is a tactipiarl. 
^ C.^ The la^^ is a f ailiire,- 

^ Code Stimulus: FEM = B (lawyer) 
GPNUT = L (jury) 

. • FEMREF = ,2. (lawyer, she)"; 
' / , ^ isPNUTREF =""X (jury) /r 

MALROLf'.= 1 (lawyer)' . 
r NUTRDLN© = .J. '(^ury) / 

^ Code Stem: FEM =* 1 (lawyer) f 

, FEMREF « 1 (lawyer) 
m^^f =1 (lawyet) 

.' ' ■ -ins. 




-Code Each Response (A~C) : FEM ~ 1 (lawyer) ' - ' ' 

^ ' I FEMREF = 2 (lawyer and persuader, 

ot tactician, or failure) 

MALROLF = a ^(lawyer). ' - \ S^. 

***ROLF = 1 (^ersuac^r, or t^cticii 
or failure) 

/ ' ' ^ I ' ' ' '* 

Rule a. When forms of the verb "to be" ar^e used, sex is cairried through to the 

yeferences, even when the sense of the -sentence might be iiit©rpreted to 
indicate otherwise. r 7 . ' ^ 

It She is a lawyer. 

% Code: . FEM « 1 (she) ' ' 

' - FEMREF «= 2 (she, lawyer) . 

Example 2: She wants to. be a lawyer. 

Code: FEM f 1 (she)' 

FEMREF. -2 (sh^, lawyer) ^ 

" h i f * ' ^'^^^■'"^ ^* never wants to become a lawyet./' jj 

'jiJs ^ * Code:* FEM = 1 (she). 

. j^"- . 'fEMIEF = 2 (^e, lawyer) 

•/ ' ExAMple 4:. She pretended she Was a lawyer, i / , 

" ■ , -. ' Code: FEM=1 (she)- . ^ /' ^'^^ " . " 

^ ' FEMREF = 3 (she, she, »Law>ejr) '^X 

Counting Distinct Entities and Reference's " ; ^ ' > ; | . , 

Rule 4a. If the number of individuals in a group Is .defined, or an' actualfnumber fs' 
^ given in the text, code the actual' number, of infembers in the Iniiividuals 
categories; not in the (hroup categories. , ' . . ^ 

Example 1: The twins had matching outfit^-. , ^ ^ i > . 

^ Code: NUT =.2 <twins) ^ * ' , ^ . . _ ; 

Example There were "^eight orphans wW nke4ed homes.. 



Code: NUT =^8 (eight orphans y 

<i ■ 

Example 3: The sextet sang "Jingle Bells." 

Code: lJUT = 6 (sextet) |, , 



EBs|C"^°°®"^^' ""^^^^^ ^ determine whether *** =» FEM, MAL or NUT 



> 
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Rule 4b, 




ule 5. 



Rule 6. 



Wh'en <iounting references in these casei, count the references as 
references to Groups , not Individuals , 

Example 1: The twins had matching outfits. 

Code: 'GPNUTRfiF = 1 (twins) - 

Example, 2: Inhere were eight orphans wfio ^leeded homes. 

cide: GPNUTREF = 1 (orphans) 

Example 3:" The sextet sang "Jlnfel^ Bells." ^ 

• Code: GPNUTREF = 1 (sextet) ' ' . ' 



Code anyone/body, no one/b^ody mid someoneVbody as one individual ^ 
(NUT = 1 ). Code everyone/bddy^as one group fCPNUT « 1) i 

Code* collective nouns* referring to people as being neutral groups 
(GPNUT). ^ - ' - ^ 



M Examplejl:. John was the only person in^the audience who was listening. 



. - -^odei . MAL = 1 (John) - 

GPNm>= 1 (audience) 

» 

Example* 2: The unioii votes to strike/ 



T'" 



BUT COMPARE: 



Code: GPJlUT = 1 (xxn±on)f^^>^ ' 
GPNUTREF = 1 .(uri^^p?-;?; 



Example 3: Western civilization is' decadent!^ 



Code: , 0 



7n 



,The difference between examples 2 and 3 ±§ the focus o^ ^groups which can 
'perform actions as a unit when counting distinct entities. The assump- 
tion made is that anions can act as a groui^ while western civilization , 
caim^i ' I '"^^ ^ 

Rule Only ^subjects or direct objects determine the number of people or groups- 

at|?^tem. When linking verbs are lisBd in a sentence, the predicate 
ho^^ps not coded as another individual or a group. 



(Example 1: The wives^ were secretaries. 

Code: > GPFEM = 1 (wives) 

GPFEMREF = 2 (wives, secretaries) 



•gp^Q Table C for a list of collective nouns, P^8^. 



Example 2; Both men and wonven »re , elementary seh^ol teachers. 

, ^ Code J GPFEM = 1 (women) » V 

' " ^ ' . GPMAL = 1 (men) . • • . • , 

^ . * GPFEMREF = 1 (women) 

. • CPMALREF = 1 (men) ' - 

GPNUTREF = 1 (teachere) * ' • • 

Frequeafly the predicate noun is ^' tole whi^Ji describes the actor(s). 
The descriptor itself is not an addijti?Qnal distinct 'entity. , 



Code, animals as indivi-duals or groups » as references, ««d In, roles (If applicable) 
when their appearance is personified or if they are anthropomorphized. 
Naming an animal implies anthropomorphism. 



^Example 1: The mother rabbit put all her baby bunnies to bed. ' 

■ /, ■ : 



•Code; FEM = 1 (mothier rabbit) 
'GPNUT = 1- (baby 'bunnies) 




FEMREF = 2?(mother rabbijCr,^ her) ^ 
\ GPNUTREF>^= 1 (Biby bunniWs) " 

' * ' FEMROLF « r^(^ther) . " ' ^ ; - r 

NUTRDING ^ 1 -(baby bunnies) ^ ' 

Example 2: The^r,o6ster-^ecked scornfully at ^the .fence. 



; Gode: - MAI< X: (rooster) \ 

MALREF 1 ^rooster)' 



Example 3: Spot wagged his^t^l. 

, ^ • Code:; MAL « 1 (St>ot) 

MAJ.REF.= 2 (Sppt'^ his) 



If animals are not persirfnified c^^anthropomprphized, code, only 'their prohoua 
references. Pronouns are. cpd|4-Befeause frequently animals are sex^tj^jed^ ' 

E;^aijq>le 4: The dog buried his bone. . t ^ » ^ 

Code; MALREF » 1 (his) ' \ ^l: ^ * 



Example 5 J 
Example 6: 



tions kill in .order to get their rfood. 
Code:. GPNUTREF = 1 (their) • 
The deer ran into the fores t. 



. Code:, 0. 
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Rule 9. J)o not -code inaniiaate objects of their references jjnlegs they are 
named or anthropomorphizjed.* 



Exasiple' 1: The fiunily safe was well hidden.' . , ' , 
t Code: 0 . " ' . ' ' ' ^ ' 

Exaaple 2: The bomb made the ^^nemy weapon^s al^solete. 
Code: ' 0 * 



.„.Example-4: Mother said, //Put G.I* Joe away and cpniS^ to dinnei^/* 



Code: FEM =' 1 (mother)" ' / ' 

MAL = 1 (G.i. Joe)\ " ^ 

. ' FEMREP = 1 (mother) ' ^ . ^ 

MALREF = 1 (G.I. Joe) . v ' ^ 

^ ' * \ 

' . " FEMROLF = 1 (mpther) 

. - . < 'mALROLM = 1 (g'.I. Joe) ' ' . _ 

- Example 4:. ^Ra'^gedy Ann is sleepy. ' \ . - 

' Code : — FEM » 1 (Rag&edy Ann) ^ ^ 

j ^ FEMREF = 1 (Raggedy Ann) ^ . • , 

Designating Roles ^ i . ^ V- 

Rule 10. Noiins which are inherently sex^lrJced (see Table A, page 2^1) ^are coded 
as both Individuals or groups and*as roLe^j^JLn themselviis. This rul« do 
not apply to pronouns. ' 



Example 1: The woifen went to work. 

• ft * » 

Code: GPFEM « ! (women) 

• GPFEMREF « 1 (women) 
^ . EEMROLFG = 1 (women) 

Example 2: The brothers were mischievous. 

. Code:^ GPJIAL - 1 '(brpth^rs^T 

/ . ^ • GPRalREF « 1 (broth«s) 

MALROLMG r 1 (brothers) 

"'Example' 3: She'^'C^t to' work. ^ ' - 

Code: FEM = 1 (she) 

^ ^JMREF = 1 (she) . 
FEMROLF « 0 • ' ^ 



Rule 11. 'When a role i's^usfed as -a name, it is coded as a roleifr 

V||i^ Exaiaple' 1: Dad, went 'to work. . • • • 

■ d . . / Code: .-MAL 1 (Dad) ^. ^ ... 

ERJC . ' ifllLKEP - 1 (Dad) .117 

ygBMO HALROLM » 1 (Dad) 



^Exaihpli*2t Dan went, to work. . ' ' ' ^ 

Co^ 'mAL-= 1 (DanT 
• MALREF = 1 (Dan)-' ' 

MALROEM tt--.' ^ • 



BUT COMPARE: 




? 



Rule 12. When a sentence contains only^ roles and .no tactual 'individuals or' gtoups, code 
• the Voles in the total male roles ^or- total female roles, and total roles 
, categories. Since -such sentences do not. contain actors, distinct 'individuals 
. and groups are not coded. ' • ^ • . ' 

_ . ' . studying to be «in ec^wiomifst or- a- physicist* is l\ard. " , • 

^ Code: TOTMALROL- = f "(economist,- physicist) • 
, ' 'TOTROL = 2 (ecefiomist, phygicis.t) . 

Example 2: Careers combined with motherhood can be exciting. 

Code: TOTFEMROL = .rv.(motherhood) 
■ TOTROL = 1 (motherhood) ■ ' ' 

Exam{)le -3: * ■ « ^ P ' ' ' . ■ - ' 

Stem: * ' ]Which word best deecrjbBs^. s^pop 

Responses: A* gossip . 

B. ^ siiob 

C. saint^ 
D* meddler 

. ' ^ "V • E. ' friend * 

Code stem: TOT:mir'^'^l^ 

Co<^e each response (A-E) : tolROL =i I (gossip or snoB/ or saint/ . ' V 

• . ^ , . cm: meddler, or'*fifieiid) , 

Rtti^ 13. When the subject of a sei^ence i^ a .group .which is subsequently described 
in role terms, count only the group or groups vhich- is/arfe the subject 
of* the sentence. • . . -.1.. > ^ 

^ Example 1: The class was made* up of nurses -and doctors. 
* ' * ^^^^ * • 

, . , Gcjde: . GPNUT = 1 (class') ' 

y ^ ' , ' GPNUTREF » 3 (clpssj ^lurses, doctors) ■ 



'FEMROLN'G = -I'- (iiurses) ' _ 

MALROLNG =? 1 (do9tors) ' . - 

'Example 2; Have you ever wondered \»hat "so^t..of people- become, volunteers? 

O Code: NUT = 1 (you) •* . • 

ERlC - > GPNUT » 1 (people> J^J'g " . ' V' 



NUTREF = 1 (you) . 
, ' ■ GPNUTREP = 2 .(people-; volunteers) ,^ 

• NUTROLUG ^ 1 (volunteers-) 

] ' 

Rule 14. Collective nouns do not carry roles.. A role is defined as a set of 
expectations, .beliefs, and b^aviors that govem'the action of-an . 
individual in a socially defined position. Thus, only individuals 
have roles. ^ • . ' • 



Example 1: .An actre^Ss "was selected from the delegation* 

Code:. FEM'= 1 (actress) 
. GPNUT - ]? rdelegation) : , - 

FEMREF = 1 (actress) 
GPNUTREF = / (delegation) 

FEMROLF = 1 (actress) • ' 
NUTROLNG «= 0 



•^Exam ple 2; Thc - imlon voted ^. to st r ik e. 



• . Code: g>NUT = 1 Cunion) , - , \ * 

' ; . GPNUTREF = .1 .(union) ' . ; ^ ' 

NUTROLNG » 0 * ' ' ' , ' < 

Misce llaneous Rules • * ' . ^ \ - 

/ — r ^ i I . ^ ^ \ ' 

Rule 15 • ^en the stem and responses form a complete sente^e but aje incomplete 
^ in themselves, code the stem' as if it' would be coded if it were the 
beginning of a complete sentence and code each response if^ it would 
be coded if. it were at the end of a complete sentence. ' . , . 

Example 1: ^ * . ' ' 

•Stimup.us: Although the lawyer sploke forcefully , * she "did not sway 
the jury. ' l 

Stem: The^ lawyer was: - ' ' . 

Responses: ^ A,' a persuader B. a tactician C a failure 

' Code stimulus.: FM^^'*' 1 -(laywer) . ^ ^ " . . 

' ' ^Smn « 1 (jury) \ \' ' ^ 

FEMREF - 2 (lawyer^, she) ^ , I t 

^ -GPNUTREF = 1 (jury) 



Q .... MALROLP » 1 (lawyef)^ 

ERIC ' ' ' NUTfeOLNG « 1 (jury) -J. lb* 



4 i ^ 
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, Code stem: FEM = 1 (lawyer) \ . , . 

V *■ ^ . '5t:MREF'= 1 (lawyer) ' ' . ■ , ' 

— MALROLF = 1 (lawyer!) ^ ^vt^ ' ' 

Code each reponse (A-C) : , FEMREF = 1^ (persuader, or tactician,' 
' ' or failure) 
.^^ • ***ROLF = 1 (persuader, or tactician, 
.f.x: - ' or failure) 



Example 2; 



Stimulus: john went to work early so he could finish his 
MSlghment. ' ' 

4 



Stem: 



Johk'Ts: 



Responses: A. a hard woVker 

B. class president 
^ Gi late for school 



Code fi^stimulus : MAL = 1 (John) 
- \ MALBLEF * 3 (John, he, his) 



Code stem: 



MAL' = 1 (John) 
MALREF = 1 . (John) 



Code Response 1: MALREF =1 (worker) • ' 
. ' NUTkOLM =r^l (worker) 

Code Response 2: - MALREF = 1 "(ciass president) 
' JNUTROLM = 1 (class_Erefiident> 



* vCode Response 3:0 



7 



Rule 16. When man is not preceded by an article and is used iA the generic it 
is; coded as GPNTJT. In this usage, aan does not have a role. Manklrid • 
is also .coded as GPNUT, and does not carry a role. [This rule is an ^ 

• ■ wn^^«''nF^^^^f^°'' sljiplify coding.] In an inplicif analysis » 

wotfds of this type would, be considered as male. . 

Example 1: In one very important sense, man has suffered as a - ' 

^ ' ' f result of his technological sophistication. 

Cod^: GPNUT =1 (man) 

MALREF = 1 (his).' " ■ \ 

GPNDTREF =« 1 (man) , * 

.' MALROLNG =0 ... 



*** 



Cbnsult Table -B to jdetertafife whether *** = FEM, MAL, or NUT. 



Exainp3|e^2^ J^kind has suffered because of trfechnology. 

Code: GPNUT = 1 (mankind) * ^- . 

GPNUTREF = 1 (mankind) . . 

^ ^ ' ' MALROLNTG = 0 

BUT COMPARE: ' ^ ^ . ' « 

Example p: A man becomes alienated when he ^immerses himself 
^ ' in the world of t^echnology'' and autbmation. 



Code: MAL = 1 (man) ' . - ' - '^xti 

1 MALREF^= '3 (man, he, himself) - 
' \ ^ ,^^^ROLM = 1 (man) 

r 

When pauns xnr pronouns are lised ag posseVsiyes, code them as persons 
or grqup^s, refereneesT^and roles if applicable. 

Exaople 1: Her purse was stolen. 

Code: FEM^ 1 (her) ' ' 

FEMREF. = 1 (her) ' . « 

f 

^Example 2: The boy's mother was angryl » 

Code: ^FEM = 1 (mother) 

MAL = 1 (boy*s) ^ ' 



^ ' FEMREF = 1 (mother) 

^ ' MALREF = 1 <boy*s) ^ 

. FEMROLF =^1 (mother) 

MAlROLM «'1 (bpy*s) 

1^ • ' 



Example 3: The family *s. safe was hidden. 

. Code J GPI^UT' = i (fafflily's) . * 
^l" ■ GPNUTREF » 1 (family's) 
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U?E _OF THE CODING SHEET AND MNOTATION SHEET 



This lection defines special rules "and procedures which are specific tp the 
^tudy,' Performance Consequences of Sex JSias i^i Test Itemis. The first part,'^ ' 
t>f this section concerns procedures used on tflhe coding §hedt and the second 
■'part ^als with special coding rules. * ' . , ^ 5 . . • 

Coding sbfefet> , * »' . . : 

^^^^Il^h^age 24 is a sample of the coding sheet used in this study .-'^'The 
^per left-hand comer contains a bo^e* f or coder initials, Bene^tl^ , this box 
is a bojc for identif icatioh of test,* subtest and prage of the test feeing coded. 
• ' ! ^ . . . ' . " ^ ^ ^ 

Xfte co4er begins coding with'box one (1) under the General Information 
Section^ The item information in the General "Information section (ID, SECTION, 
CONTEIlT, ITEMID) is recprded only once for each item.. The first line 
f ollovrLng the General Information section contains the codings ^n- 35 content 
categories for the stimulus of an item.' The Second line contains the codings 
in 35 Content categories for^ the stem of an item. The. third line cQntains * . 
the codings dn 35t content categories for Response 1 of an item. The ^.ourth 
line contains the^ codings in 35 content categories 'for Response 2 of in, ^ 

Item. The fifth line contains tlfe codings in 35 content categori es for- 

Response 3 of an item. The 'sixth line contains the codings in SS^^content ' 
categories for Response 4 of an item. The seventh line contains thd. codings 
in 35 content categories for Response 5 of an i«tem. /.The eighth' line is ^ 
Ipft blank and will later Be filled ;Ln with data on performance on the -item. 

There is ^^pac^jjto cod^ two separate test .items per, coding sheet page. . 
For future instruction as to the information ,4Qded into- eacfo individual box 
on the codinl^ sheet', refer to pages 39-48i * .' - ^ ^ 

^ Special Coding Rules ^ - f*',^ / 

• *• , • • * ' V 

1. Each coder keeps an Annotation Sheet on which irregularities * 

and subjective coder decisions are noted. Tfie item in- p 
question is identif iedj>y the numbers which appear in the- ^ 
general information sections arid notations or comments follow.. 
See. sample of an Armotation Sheet, page 25. 1 ^ 

2. In the Distinct;*Entities^ mid Rgfer^nces sections there is, room _ 
' to ei^ter double^ digits wheti counting. In the Roles section,- 

' there is only -room ;to enfer one digi-^t per role type* If the • 

* * number to be entered is greater than 9, cpd^ 9 inr tlie box ,on 

-the Qoding sheet, and i^ote the actual number on the Annotation. ' ^"^'^ 
Sheet. V ^ . , ^ * 



3. Note ^all it ems. on the Annotation Sheet which hav^'tdie word 
mankind or the generic maiv in them. -See^Rule 16, page 21. 

- ' • , • • 

4. Whenever the coder manias an arbitrary or subjective decision 
.^*in >5Qding an item^ the decision should be recorded on the 

Annotation Sheet. . ♦ ^ 



Coder iniciais: 



stimulus 



test: . 

subtest: 

pages: 



General 
Info 



o 

M 

w 

CO, 



* 1 » 



I t t I I 



(stem) 



(res^ond^ ) 



(re'&ponse2) 

\ \ * ' » 



(response^) 



(response'^) 



.^(responses) 



• I'll 



stimul^as 



ERIC 



{[stem) 



(tespofese h 



-( response 2) 



(response 



PERFORMANCE CONSEQUENCES OF SEX' BIAS IN TEST ITEMS 



DISTINCT. ENTITIES ■ 



People 



^1- 



1 ' 1 



1 ' \ 



J : ■ I 



t t I 



Groups 



REFERENCES 



People 



(14 



(14 



i J. 1 1 



\ \ \ \ 



\ \ \ \ \ 



J—J— L 



I t t ! I 



' ^ * ' ' 



(14 



J» T 



(14 



4 . i 



1111! 



I ! 1., 



J— L- L 



JL-U 



J L 



1 ! t 



! : h I 



(response 

1 



(response. 
I . I • - ' 



X-i—L 



! ! I ] ! 



1. V ! 



1 ^ 



1 I 



! ! \ 



\ !. 1 



! : f 



Groups 



(14 



(M 



3 •» 



(14 



O O 



t I t t I 



I ! 



t I I t I 



\ y ' ! ! 



] I I I I 




ROLES' 



People, 



(14^ 



9 2 



^ 9 o 

OS X 3eS 



O O O 

, . . . 

QlQ O 



i ' 



/ 

\ I « 



I 1 



I K. 1 '.1,1 ! !. 



t « > ' I 1 ^ ' 



Groups 



O 



i 



O 
H 



oil 



I I I I 



J I I I lU L 



1 t ' « i » t 



^ I t I I t t 



» t I ■ t j. ? . t i « « . 



I ! I I I 



' I ' i I 



CO 
CO 



I 

ro 
it^ 
I 



I I ^ I ♦ « • ^ 



! I \ \ i ! i ' I 



> f < ' L 



t ' » ' I ' 



I 1 * I ' 



I t . I I 



i I . I I 



I I ' , 



t « ? t I I 



> r I t t I I 



1 I 



» ' ' i t 



till 
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SAMPLE ANNOTATION SHEET 



It' 



Item / / 

5 

638^01 
638005^ 
638021 



Item 
Component 

Stimulus 

Response 1 




Notation 

The 9 'Co4ed in FEMROLF equals* 13* - ^ ^ 

Man (generic) is used in this item. 

Coded braggart as NUTKOL, role not found on Table B. 



C 

r ' 



ERLC 



■ } 
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TABLE A 




Vords which are inherently sex linked- that are automatically coded feaale 
or male plus words which are defined by the dictionary^ as sex specific. 
(These words are also .automatically coded as roles, see Rule--10, page 18.) 



J* 



Female 

- ' - 


■ Male 


act'tegs 


boy. 


aunt 


brother * 


* ballerina 


businessman 


•bride 


. cowboy. 


daughter 


Dad 


duchess 


duke 


girl 


father 


grandmother 


fireman 


lady 


grandfather 


Mom 


c husband 


mother 


king 


ni^ce 


* 

xora 


LIUlj. 


lUciXXlugtl 


- princess 


male 


quei^n • ^ 


man 


sister - 


nephew 


stewardess 


- prince - 


wife <^ 


^alesmau 


wom^n V . - , 'f 


• son 


*. 


tycoon 


* 


- uncle- . 








*T^e Dictionary used is The,Ran(fom House Dictionary o& the English Language 
(The Unabridged Edition), 1967. 
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TABLEp 3 



Roles which have been designated 'as i^emaje roles, male roles or neutral roles • Roles 
^were referenced from the occupation fist (see page* 34) . When 80% ^or more of the indl- 



vidual^ engaged in an occupat 
role.. Other occupations, were 

. Female Roles 

airline stewardess/flight 
attendant 

aunt . ^ ^ 

daughteTT" - \ 
duchess. 

elementary teacher 
female 

grandmother 
hairdresser 
^ady 

librarian 
Mom 
mother 
niece 

nurse ^ , * 

office worker 

princes 

queen 

receptioi^ist 
, secretjary 
sister 

wife ^ ; 
woman 




I 



&ie one sex, the occupation was defined as a sex-*typed» 
y^^ified' as neutral ^roles. , • 



Male Rol 



airplane pilot. ^ • 

ambassador 

banker 

bank officer 

bartender 

Board of Education member 
boy^ 

brother • . ^ * 

businessm*^ 
clergyman, minister 
college president 
craftsman 

Dad ■ V. ^ 

dentist ^ 

du^e ^ * 
economist 
engi^ieer » 



farm labprer/fairmer 
father ' ' \ 

• ' r • 

fireman , j 

» foptman 
f or^sf er , ; conservationist/ 
lumberjack, firewatcher 

^general ^ ^**^v^.„ 
geologist \ 
geo'physicist ' '* 
governor, 
grandfatfier 
Grandfather 
feuard , watchman \ 



Neutral Roles 



accountant 
- actor 
anthropolo^st 

archeologist 

/ . 
athlete 

author 

biologist ^ 

clerk 

coach 

'1 ^ • < ' 
college professor 

curator 

editor, and reporter 

elementary and s'econdary 

' schgol adminis4:rator 

garage worker, /gas 
station^attendant ^ 

inspectori ' ^ 

niusician 

painter, sculptor 
psychologist ^ ^ ^ 

psychology profes^pr 
salesman \ 
sociologist;^ * ' 

sociology -prof essor 

thief J ; : A' , ' 

I 'I 
union meiAjbfer 

volunteer' • ' • 



ERJC 



.12 



r 



Roles 



V 



^ TABLE B (Continued) 

Hale Roles < 
r 

historian , 
Jiusband 

janitor /building- custodian 
jeweler ' , ' 

judge - 
king 
laborer 

lawyer / attorney, 
) ; l^rd ' ' • 

mailcarrier postman^ 
male 
^man 

manager^ 

maypr . ' 

nephew 
pharmacist 
philosopher 
physicist 
' . fphysicria^, doctor 
plumber 

policeman and detective 
postmaster '^^ 
prince 

/principal ^ 

railroad condufctor 
^representative 

robber • * 

ruler. ^ / • . 

scieatiS't 

senator 

son 



Neutral Roles 



/ 



iir-- 



] f \ 
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Female Roles 



TABLE B (Continued) 
. - Male. Roles- 



taxicab driver 
ti^icle 

veterinarian 



(^. JSeutral Roles 



ir 



'I 

■M ■ ■ - 

Roles which were not on the occCipation list but appeared in the' tests. The roles were 
generally classified as neutral roles except where otherwise ^in^'icated - 



Female Roles 



Male Rolgs 



alchemist 

Captain 
• city leader 
. city 6;fficial 
^o^^ftel 

dictatot^^ ^ 



^^nian 



Lieutenant * 

Little League Manager 

marine 

mayoralty candidate - 

member- -oE the New York 
~ Stock Exchange 

TiTerchant ^ ^ — 

political candidate • 

politician' . 

Secretary of the Interior 

Ser|eant * , ' 

tycoon 



V 



Neutral Roles 

airline employee 
amateur 
apprentice 
ar^ collector 
artist 
assistant 
bore ^ 
boss 

cartoonist 
citizen 
^ cobbler 

colonist ^ 
i comedian 
connoisseur 
'council member 
poward 
critic 
demon 
employer » 
etiemy 

e:iperimenter 
expert , . . 
,explQrer 



I 



\ - TABLE B (Continued) • , ■ 
Feamale Roles * Male Roles ^ • • Neutral Roles 

, * . ^ factoty worker 



ERJC: 



/ fisherman 



9^ 



* V 



fighter ' J 
fisl 
foe. 



friend 
fugitive- 
gangster 

hawker » * 

hiker 

hobo . 

hunfer - 

immigrant 

institutional investor 
intellectual ^ / 
invader ' 

jui*or ^ ^ 
magician: 



missionary 
iiohopalis t • 
inoforist 
observer 
'officer ^ ' 
orphan 
outsider, 
owner 
, paint seller 
passenger < 
peasant 
pewterer 
- pioneer 
playe;r 
poet 
^r^ent 
pupil ^^ .^.^ 
13Q /' \ ranker.- 



. • . ' . reader 



-t ^ ' • . 31 - ' . V ' 

TABLE "B (Continuea> 

Female Roles Male Roles . Neutral Roles 

* . . ' . ^ ' ^ referee 

* ' ' * — * • ' reformer * N 



1 -'i 



it 



\ 



- ^ • rogue 
^ ^ schol.ar 

servant^^^ • 
settler 
'^".7; * skater 
* 'Skipper 
^ ^ slave 

socialist 

r ' ' , 

^ sorcerer 

specfalist 
f speaker ^ , 
^ •■ student 



V 



supefvisorv. 

, ' villain ' 

' %iolesale -furniture 
" dealer 

^ worker 




List o£ 



peseafatl^ve .Colle6tive ^louns • ' 

»'•'** 

' • , ^ : -i"; ^• 
army , . ^ ^ 

delegation ' \ . > 

> organization ^ 
labor '^^^ - • 
management ► , 

^ corporation ^ 
cQmmunity 

culture " " ^ 

civilization . - 
society « ' , ^ 
colony ' " * 

empire 

; population , ^ ' ' • 
audience^ ' . 
band ' * 

class ' * ' c 

■ ^ committee . ' \ 
company 

^ gang^ / 
^ y group . ; 

majority 
mankind j ^ 
team ' ^ \ 



J 




Test Identification of Subtests Coded for 
"Performance Consequences of Sex Bias^'in Test Items" • . 

Numbers, assigned follow the order 'of the General. Information 
Section. Test ID is the first number. Section of test is the' 
second number. Content of the section is the third numbers 
[Item number is the fourth through sixth number and not giVen 
in this »list] . / \ ' 



STEP (Series II) Tests 



411 = Reading, Form 2A, Part I 

421 = Reading, Form 2A, Part II 

412 = Mechanics of Writing, Form 2A, Part^ I ^ 

422 = Mechanics of Writing, Form 2A, Part 'II 
432 = English Expression, Forn^A, Part I • . ^' ' 

^ 442 = English Expresf igla, ^Form^SA, Part II 

^ 413 = Mathematics Basic;,.Con<^pts ,. Form 2A, Part I 

Mathematics Computation, Form 2A is not being coded, 

r o Hf \ 

414 = Social Studies, Form 2A, /Part 
424 = .Social Studies, Form ''2A, (Part II 



41'5 = Science, Form 2A, Part- I 
425-= Science, Form 2A, Part II 



iAt (Form F)^ Tests 



331 = Primary^V Test; 3^ Redding, Part AT Sentences \ 
341 ^''primary I,^Test:3, Reading, Part B: Sto&ies 

371 =^primciry 11, Test 3, Reading, Part Aj^Sentences 
381 =^rimary II, Test 3, Reading, Part B: Stories 



/ r 

ITBS (Form 6) Tests 



211 = Test V, Vocabulary (levels 9-14) 

221 = Test R, Reading Comprehension (levels 9-14) 
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212 ='Test L, 

222 ^,Test L', 

.232 = Test L, 

2A2 = Test L, 

216 = Test W, 

226 = Test W, 

236 = Test W, 



Language Skills, L-1: 
Language Skills, L-2: 
Language Bkills, L-3: 

Language -^kills , LrA : 

t 

\^ 

Work- Study Skills ^ W« 
Work-Study Skills, W- 
Work-Study Skills, W- 



SfTelling (levels 9-14) 
C'Spitalizaifion (levels 9-14) 
•Punctuation (levels 9-14 ) 
Usage (levels 9-14) 

1: Map Rea^ding^ (levels 9-14) 
2: Reading ' Graphs and Tables (levels 9-tr4) 
3: Knowledge and Use of Reference Material 
• • . (Levels •9-l4)t 



213 =*Test M," 
, 223 = Test M, 



Mathematics Skills, M-1 : Matliematics. Concepts (levels 9-lA) 
Mathematics Skills, M-2: Mathematics Problem Sojvinp, 

• . (levels 9-lA) 



CAT- (Levels 5, Form A) tests, 1970 Edition 



113 = Section 5, Mathematics Problems 



-4 




i 13 



Occupatfon List " ^ . ' 

The idejitif icatioil of particular role^.as f*emale,. male, or neutral was decided 
by the percentage of females- and males found actively engaged in that role as"^ 
documented by several different sources. ' The following is a list of oconpa- 
tions found in the tests being coded, the percentage of" females actively 
involved in that occupation, and tte sou^e of ' the^ercentage. The primary 
sources and the^ir nota^ioDs appear below. Other sources are listed along with 
the occupations. <, . . - • - 



* Occupation Characteristics » 1970 Census of th^e 
Population , "Summary of Sacial and Economic 
Characteristics' of the experienced civilian labor 

^ force by /detailed occupation and s^: 1970". 

^ ' V . \ 
World Almanac, 1975 



Occupational 'Outlook Handbook (1974-75) U-^ S. 
Departm&it of Labor, Bureau of Labor Statistics, 
d975 Bulletin. ' ' • ' ^ 



Occupation 



actors ^ ' 

aiijpl^e pilots/aviators 

airline stewardess /flight atten4ants 



ambassaxlors 
anthropologists 



Percentage of. 
. C6 Females 

39 

.^^ ' 1 

95 " 

• \ 

1 ^ 

53.8 



arch^eo logis ts 



athletes^ 

authors • ' ^' ; 

bank officers/bankers 

bartenders. 

biologists- 



ERIC 



26. '83 



27 
•29 

20 

.34 



136 



Source 

- f 

* 

Monthly Labor' Review , 
Nov.vl9?5, Vo:yj. 98, #1. 
Proportion of Doctorates 
Earned by. Women, By^Major^ 
Field of -Study and Sub- 
field, United States, 
J.969-1972' 

Monthly Labor f Review , 
.Nov. 1975^ Vol. 98, //I " . 
Proportion©qf Doctorates 
Earned by Women, By 'Majol:* 
Field of Study and Sub- • 
field, Unit^ States, _ , 
1969-1972 • - • 

• ■ * 



C ■ - 

Occupations * 
Board of •Education Members 

clergymen/minis ters 
clerKs 



Occupation List (Continued) 

Percentage of 
Females 

> ^ 12 • 

\ I 

2 

25.2 



coaches 

.college presidents 
cj;^ftsmen/cr^f t workers 



curator s /^rchlvis t§ 
' "dentists , • ! 

economists 

editors and reporters ^ 
elementary ,and secondary teachers 
elementary teachers ^ 
♦ engineer^ ♦ 
•farm' laborers/ farmers 

firemen • 

c ^ 
/- foresters, conservationists/ 

lumberjacks, firewatchers 
. garage workersj,. gas station attendants 
generals 
*~ geologists 
. geophysicists 



25 
5 



4'.5 



30 ^ 

3 
IX 
40 
69- 
83 

1 
16 
♦ 1 

- * 

3" - 
28 
1 
3 

2.91 



^ERIC 
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Source 

■ - 

Women on Schoolr^ards ^ 
National •Sdhool Board ♦ 
'Association Report -•#1974-1. 



Monthly Labor Review ^\ . 
Nov. 1975, Vol.- 98, #1 
Table 6, Employment by 
occupation t**. total and 
^ women workers, X962-1974 



Survey done in « fall *75 
of accredited colleges & 
universities by the Office 
of^^Women in Highe;r Educa- 
tion of the, American 
Council on Education 

Monthly Labor Review," 
Nov. 1975, Vol. 98, #11 
Table 1, O^upational 
distributl|§ •of employed 
women, 196^-1974 



* 



ic 

icic 
ic 



Monthly Labbr Review ^ 
-Nov. 1975, Vol. 99,. #i 
Proportion of Doctorates 
Earned by .Womeji, By Major 
Field of Study' and Sub-* 
'field. United States,' 
.a969-l972 




^ ^Pccupatlons . 



governgrs 
guards 

hairdressers 
historians'' 



inspectors /checkers . 




1 



janitors 
j eweiers 
judges • 

lawyers /at t orney s 
mailmen/pos tmen 
managers/managerial 

- V ; 



Mayors 

♦ 

musicians \ • ^ 

nurses • ^ 

office workers /clerical 



A' 

painters, sculptors 



pharaacists 
philbsophfers 



physicians/doctors 
physicists* 
plumbers^ , ^ * 



25 
11 

4 , 
4 
.8 

18.5 



35 
97 

68.8 




35 
11 " 
' 6 

9 

3. 



Source 

* 
* 



Monthly Labor Review , 
Nov. 1975, Vol. 98, jtl 
Table 3, Occupational 
.participation rates: 
women as a percent of the 
total employed workers, 
1962-1974 



* 

* 



Monthly Labor Review , 
Nov- 1975, Vol. 98, #11 
Table 1, 0ccupati9nal 
distribution of employed 
women-, '1962-1974 

6.S. Conference of Mayors, 
1120_Eye Street, N.W,, / _ 
Washington, D.C. • 



Monthly Labor Review , 
Nov. 1975, Vol. 98, #11 
Table 1, Occupational 
distribution of employed 
women, 1962-74 



Survey. done in 1974 by ^ 
the ^American Phllosophi^ial 
Association 



* 
* 



id 
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Occupation List (Continued), 



Occupations 

policemen and detectlyes • 
postmasters > I 

professor? (college) /faculty 
psychologists ^ 
psychology^ professors 
'principals 



Percentage of 
Female^ 



elementary 
jr. high- 
sr..|iigh 
total: 



railroad conductors 
receptioa^-sts 
representatives (House/ 
robbed/burglars 
rulers /heads ofScountries 
, salesmen ^ i 
scientists' 
secretaries 
senators ^federal) 
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